test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-07 14:29:47 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
20,507 Commits 96 Branches 105 Tags 188 MiB
c0200eec39
Commit Graph

156 Commits

Author SHA1 Message Date
yah01
70995383bf
enhance: modify log to avoid ambiguity and improve readability (#28331) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-10 14:32:20 +08:00
yah01
ece592a42f
Deliver L0 segments delete records (#27722) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-07 01:44:18 +08:00
yah01
90e2c63d9e
Fix getting incorrect CPU num (#28146) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-11-06 06:02:16 +08:00
yah01
dc89730a50
Support collection-level mmap control (#26901) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-02 23:52:16 +08:00
congqixia
852be152de
Change task sourceID to stringer interface (#27965) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-27 01:08:12 +08:00
wei liu
e0222b2ce3
refine target manager code style (#27883) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-25 00:44:12 +08:00
yah01
be980fbc38
Refine state check (#27541) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-11 21:01:35 +08:00
smellthemoon
a0ca982a52
Fix typo in priority name (#27558) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-10-11 14:19:35 +08:00
Xiaofan
41124f281a
Remove parser dependency (#27514) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-08 15:05:31 +08:00
yah01
a8ce1b6686
Refine QueryCoord stopping (#27371) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-27 16:27:27 +08:00
yah01
6539a5ae2c
Refine DataCoord status (#27262) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-26 17:15:27 +08:00
SimFG
26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
yah01
168e82ee10
Fix panic while handling with the nil status (#27040) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-15 10:09:21 +08:00
congqixia
edde3cf1c7
Add tracer for querycoord tasks (#27058) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-14 09:59:19 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam (#26725) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-05 10:31:48 +08:00
congqixia
1a8cf5c415
Organize all mockery generation commands in Makefile (#26826) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-04 21:19:48 +08:00
yah01
f9c060e0d2
Treat balance task with released source segment as stale (#26453) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-22 17:12:22 +08:00
congqixia
065d1a962e
Add sourceID output for task.String and fill reduce channel reason (#26447) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-18 13:50:19 +08:00
yah01
9723787141
Calculate memory usage without page cache (#26389) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-16 17:04:17 +08:00
Enwei Jiao
7d61355ab0
Refactor log for Query (#26310) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-08-14 18:57:32 +08:00
Bingyi Sun
54c0e64059
Fix search on empty segments set bug (#26136) 
Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-08-08 11:17:08 +08:00
wei liu
6f89620a43
remove pull target rpc from lock (#26054) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-04 10:31:06 +08:00
yah01
df26b909f0
Fix panic while saw the failed task status (#26082) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-08-02 13:47:05 +08:00
yah01
2180ef180c
Record only failed task error (#26033) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-08-01 10:11:05 +08:00
Bingyi Sun
a3e22786ed
Move meta store to kv catalog (#25915) 
Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-07-31 13:57:04 +08:00
congqixia
2b9ec565bb
Fix task executor can not dedup same task (#25901) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-26 08:51:02 +08:00
congqixia
a669440ee9
Allow querycoord executor load sealed segment with no index (#25902) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-25 18:47:01 +08:00
congqixia
efcaa07fe9
Implement Stringer for task related enums (#25860) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-24 13:57:00 +08:00
congqixia
3c503afe7c
Use typeutil.ConcurrentMap instead of sync.Map (#25846) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-24 10:23:01 +08:00
congqixia
1045c88102
Support replace indexed field in QueryCoord (#25747) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-19 21:22:58 +08:00
yah01
224515eaa3
Add segment dist containing condition for loading segment (#25736) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-19 15:02:58 +08:00
yah01
228c0df3f7
Skip record ChannelNotFound error while loading segment (#25669) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-18 11:53:18 +08:00
MrPresent-Han
49655d2f13
fix panic due to lack of indexInfo on querynode(#25362) (#25613) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-07-17 19:23:19 +08:00
yah01
948d1f1f4a
Handle errors by merr for QueryCoord (#24926) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-17 14:59:34 +08:00
wei liu
b62c82af22
fix set target version in loading sealed segment (#25603) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-07-14 20:00:31 +08:00
yihao.dai
c56a26a556
Add SetMetricType and set metric only for shard leader (#25285) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-07-04 10:20:25 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database (#24769) 
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-25 17:20:43 +08:00
yihao.dai
c7dc1c067a
Assign metric type in load segment request (#24917) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-16 16:50:41 +08:00
Gao
51404abe93
Add missing indexInfo in watchDmChannelsRequest (#24696) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-06-14 14:40:39 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
yihao.dai
5f7099a9bd
Fix search failed due to metric type mismatch (#24691) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-06 21:40:35 +08:00
yihao.dai
b0ce4cff83
Assign metric type in search request when user has not explicitly assigned (#24591) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-01 18:10:32 +08:00
MrPresent-Han
b09e7aeaf7
support detailed task metrics(#23414) (#24507) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-05-30 14:59:28 +08:00
yihao.dai
1a3dca9b5e
Fix dynamic partitions loading (#24112) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-18 09:17:23 +08:00
yihao.dai
7e0d1492c7
Load delete from channel checkpoint (#23961) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-09 19:10:41 +08:00
MrPresent-Han
b517bc9e6a
refine balance mechanism including:(#23454) (#23763) (#23791) 
1. balance granuity to replica to avoid influence unrelated replicas
2. avoid balance back and forth

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2023-05-04 12:22:40 +08:00
foxspy
6f4ed517de
add growing segment index (#23615) 
Signed-off-by: xianliang <xianliang.li@zilliz.com>
 2023-04-26 10:14:41 +08:00
wei liu
cbfe7a45ef
fix pull target (#23491) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-18 18:30:32 +08:00
congqixia
b5a73e6d1a
Add OWNERS files for querycoordv2 sub pkgs (#23489) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-18 15:52:30 +08:00
wei liu
dbbd703667
fix balance generate unexpected task (#23299) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-11 14:38:30 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
MrPresent-Han
afd874b736
enhance segment balance by considering global rowCount(##22914) (#23056) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-04-03 14:16:25 +08:00
yah01
ae0f467c02
Fix segment/channel may be re-loaded/subscribed (#22969) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-27 18:28:00 +08:00
yah01
081572d31c
Refactor QueryNode (#21625) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-27 00:42:00 +08:00
XuanYang-cn
93bc805933
Enhance ID allocator in DataNode (#22905) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-23 19:43:57 +08:00
yihao.dai
1f718118e9
Dynamic load/release partitions (#22655) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-20 14:55:57 +08:00
yah01
3d8f0156c7
Refine scheduler & executor of QueryCoord (#22761) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-16 17:43:55 +08:00
yah01
21ba8182ee
Refine task create errors (#22745) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-14 18:51:53 +08:00
yah01
1a4732bb19
Use new errors to handle load failures cache (#22672) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-10 17:15:54 +08:00
yah01
d4fccdd135
Add reason for spawning task (#22549) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-06 10:13:50 +08:00
jaime
d126f06946
Decouple mq module from internal proto definition (#22536) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-03-04 23:21:50 +08:00
Enwei Jiao
697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
wei liu
73c44d4b29
resource group impl (#21609) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-01-30 10:19:48 +08:00
congqixia
5f3d3dc4fc
generate-mockery for querycoordv2, querynode and rootcoort (#21714) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-16 13:59:42 +08:00
bigsheeper
2146af1fb2
Return insufficient memory error when load failed (#21574) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-01-10 20:35:39 +08:00
Xiaofan
d16b7c3c2d
Add more logs for debugging load balance (#21389) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-12-27 15:57:30 +08:00
yah01
396a85c926
Fix the number of executing tasks may break the limit (#21318) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-20 20:33:27 +08:00
bigsheeper
fc10c74005
Use channel-cp as seekPosition when FromDmlCPLoadDelete (#21110) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-12-13 16:17:22 +08:00
Enwei Jiao
89b810a4db
Refactor all params into ParamItem (#20987) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-07 18:01:19 +08:00
SimFG
f8cff79804
Support the graceful stop for the query node (#20851) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-06 22:59:19 +08:00
yah01
ddd29ea6ab
Optimize scheduler, increase merge tasks probability (#20922) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-01 16:25:16 +08:00
Enwei Jiao
2ecdb4ba4a
Etcd config source support TLS (#20874) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-30 18:23:15 +08:00
yah01
060649b8aa
Refactor task scheduler and executor (#20828) 
Make the performance able to scale out

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-30 13:57:15 +08:00
Enwei Jiao
c05b9ad539
Add event dispatcher for config (#20393) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-17 18:59:09 +08:00
yah01
cc371d6801
Task/Action won't finish util the RPC returned (#20669) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-17 17:55:09 +08:00
wei liu
1177ed1a33
fix duplicate watch channel task (#20667) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-17 15:05:08 +08:00
yah01
2d806788fb
Add version for subscribed channel (#20585) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-15 13:21:07 +08:00
yah01
a82c235599
Scheduler should returns err if the queue is full (#20465) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-11 17:55:05 +08:00
wei liu
eaa5cfdcb5
fix leader observer sync logic (#20478) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-11 11:43:06 +08:00
yah01
a68cec85af
Refine QueryCoordV2 metrics (#20461) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-10 15:01:04 +08:00
wei liu
f925fa7661
fix check segment stale task (#20401) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-08 19:43:05 +08:00
wei liu
c5cd92d36e
update target (#19296) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-07 19:37:04 +08:00
yah01
4112c667d0
Refine task's priority (#20364) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-07 14:53:02 +08:00
yah01
c08f11420f
Fix Release tasks may end with incorrect status (#20263) 
If the resource has been removed, the release tasks should end with
Succeeded status, not stale

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-03 15:01:35 +08:00
Xiaofan
4f6b87311e
Refine request merger (#20189) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-01 14:27:34 +08:00
yah01
0b939c5735
Check whether ubsubscribe/release task is stale (#20165) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-01 12:33:35 +08:00
wei liu
4412cfcaaf
reduce querycoord unnecessary panic (#19925) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-10-28 17:15:32 +08:00
smellthemoon
581e1d8c35
Fill MsgBase ALL (#19912) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-10-21 15:57:28 +08:00
yah01
7cf26011e6
Add more logs for checkpoint of channel (#19885) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-19 13:19:27 +08:00
yah01
c61a4d1243
Fix panic due to double stop (#19849) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-19 12:13:28 +08:00
yah01
b0404cbbab
Remove dropped segments' info (#19839) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-17 19:41:26 +08:00
SimFG
a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
yah01
146a3342c3
Use task's ID as the MsgID instead of source ID (#19751) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-13 15:21:23 +08:00
yah01
1c71844b8d
Add license header (#19678) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-11 11:39:22 +08:00
yah01
e6c7286bee
Remove task timeout (#19583) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-30 19:48:55 +08:00
yah01
2e9a08aecf
Use segment's position as the delta position (#19577) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-30 17:20:55 +08:00
yah01
9dcee37e1c
Skip canceled tasks (#19513) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-28 19:58:54 +08:00
yah01
377f856833
Fix balance may confuse leader observer (#19435) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-28 12:10:54 +08:00
yah01
dff4d1042e
Fix can't release growing segment (#19418) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-26 14:08:52 +08:00
yah01
6ba52366c5
Fix load segments can't be retried (#19414) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-26 10:54:52 +08:00