Commit Graph

79 Commits

Author SHA1 Message Date
congqixia
efcaa07fe9
Implement Stringer for task related enums (#25860)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-24 13:57:00 +08:00
congqixia
3c503afe7c
Use typeutil.ConcurrentMap instead of sync.Map (#25846)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-24 10:23:01 +08:00
congqixia
1045c88102
Support replace indexed field in QueryCoord (#25747)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-19 21:22:58 +08:00
yah01
224515eaa3
Add segment dist containing condition for loading segment (#25736)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-19 15:02:58 +08:00
yah01
228c0df3f7
Skip record ChannelNotFound error while loading segment (#25669)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-18 11:53:18 +08:00
MrPresent-Han
49655d2f13
fix panic due to lack of indexInfo on querynode(#25362) (#25613)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-07-17 19:23:19 +08:00
yah01
948d1f1f4a
Handle errors by merr for QueryCoord (#24926)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-17 14:59:34 +08:00
wei liu
b62c82af22
fix set target version in loading sealed segment (#25603)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-14 20:00:31 +08:00
yihao.dai
c56a26a556
Add SetMetricType and set metric only for shard leader (#25285)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-07-04 10:20:25 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database (#24769)
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
yihao.dai
c7dc1c067a
Assign metric type in load segment request (#24917)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-16 16:50:41 +08:00
Gao
51404abe93
Add missing indexInfo in watchDmChannelsRequest (#24696)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-06-14 14:40:39 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yihao.dai
5f7099a9bd
Fix search failed due to metric type mismatch (#24691)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-06 21:40:35 +08:00
yihao.dai
b0ce4cff83
Assign metric type in search request when user has not explicitly assigned (#24591)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-01 18:10:32 +08:00
MrPresent-Han
b09e7aeaf7
support detailed task metrics(#23414) (#24507)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-05-30 14:59:28 +08:00
yihao.dai
1a3dca9b5e
Fix dynamic partitions loading (#24112)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-18 09:17:23 +08:00
yihao.dai
7e0d1492c7
Load delete from channel checkpoint (#23961)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-09 19:10:41 +08:00
MrPresent-Han
b517bc9e6a
refine balance mechanism including:(#23454) (#23763) (#23791)
1. balance granuity to replica to avoid influence unrelated replicas
2. avoid balance back and forth

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-05-04 12:22:40 +08:00
foxspy
6f4ed517de
add growing segment index (#23615)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
wei liu
cbfe7a45ef
fix pull target (#23491)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-18 18:30:32 +08:00
congqixia
b5a73e6d1a
Add OWNERS files for querycoordv2 sub pkgs (#23489)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-18 15:52:30 +08:00
wei liu
dbbd703667
fix balance generate unexpected task (#23299)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-11 14:38:30 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
MrPresent-Han
afd874b736
enhance segment balance by considering global rowCount(##22914) (#23056)
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-04-03 14:16:25 +08:00
yah01
ae0f467c02
Fix segment/channel may be re-loaded/subscribed (#22969)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-27 18:28:00 +08:00
yah01
081572d31c
Refactor QueryNode (#21625)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
XuanYang-cn
93bc805933
Enhance ID allocator in DataNode (#22905)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-03-23 19:43:57 +08:00
yihao.dai
1f718118e9
Dynamic load/release partitions (#22655)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-03-20 14:55:57 +08:00
yah01
3d8f0156c7
Refine scheduler & executor of QueryCoord (#22761)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-16 17:43:55 +08:00
yah01
21ba8182ee
Refine task create errors (#22745)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-14 18:51:53 +08:00
yah01
1a4732bb19
Use new errors to handle load failures cache (#22672)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-10 17:15:54 +08:00
yah01
d4fccdd135
Add reason for spawning task (#22549)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-06 10:13:50 +08:00
jaime
d126f06946
Decouple mq module from internal proto definition (#22536)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-03-04 23:21:50 +08:00
Enwei Jiao
697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
wei liu
73c44d4b29
resource group impl (#21609)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-01-30 10:19:48 +08:00
congqixia
5f3d3dc4fc
generate-mockery for querycoordv2, querynode and rootcoort (#21714)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-16 13:59:42 +08:00
bigsheeper
2146af1fb2
Return insufficient memory error when load failed (#21574)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-01-10 20:35:39 +08:00
Xiaofan
d16b7c3c2d
Add more logs for debugging load balance (#21389)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-12-27 15:57:30 +08:00
yah01
396a85c926
Fix the number of executing tasks may break the limit (#21318)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-12-20 20:33:27 +08:00
bigsheeper
fc10c74005
Use channel-cp as seekPosition when FromDmlCPLoadDelete (#21110)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-12-13 16:17:22 +08:00
Enwei Jiao
89b810a4db
Refactor all params into ParamItem (#20987)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-07 18:01:19 +08:00
SimFG
f8cff79804
Support the graceful stop for the query node (#20851)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-12-06 22:59:19 +08:00
yah01
ddd29ea6ab
Optimize scheduler, increase merge tasks probability (#20922)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-12-01 16:25:16 +08:00
Enwei Jiao
2ecdb4ba4a
Etcd config source support TLS (#20874)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-30 18:23:15 +08:00
yah01
060649b8aa
Refactor task scheduler and executor (#20828)
Make the performance able to scale out

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-30 13:57:15 +08:00
Enwei Jiao
c05b9ad539
Add event dispatcher for config (#20393)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 18:59:09 +08:00
yah01
cc371d6801
Task/Action won't finish util the RPC returned (#20669)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-17 17:55:09 +08:00
wei liu
1177ed1a33
fix duplicate watch channel task (#20667)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-11-17 15:05:08 +08:00
yah01
2d806788fb
Add version for subscribed channel (#20585)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-15 13:21:07 +08:00