Commit Graph

217 Commits

Author SHA1 Message Date
zhenshan.cao
0ac4bc32a5
Add logic of autoindex (#19570)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-10-08 15:38:58 +08:00
yah01
e6c7286bee
Remove task timeout (#19583)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-30 19:48:55 +08:00
SimFG
ccf30d358c
Revert the milvus.yaml file (#19568)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-29 21:53:31 +08:00
yanliang567
3e9376f951
Disable time travel, shorten growing segment lifetime (#19520)
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2022-09-29 19:42:55 +08:00
wayblink
54e2c079c4
MEP-30:Support basic coordinators active-standby mechanism #16901 (#19166)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-09-29 18:35:02 +08:00
bigsheeper
5405dedd2e
Fix DDL rate limit unit, and improve log (#19504)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-28 20:58:54 +08:00
yah01
9dcee37e1c
Skip canceled tasks (#19513)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-28 19:58:54 +08:00
cai.zhang
c58a8d8fd6
Support bind IndexNode mode (#19247)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-27 22:38:55 +08:00
Ten Thousand Leaves
b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-09-26 18:06:54 +08:00
bigsheeper
804c18df68
Update quota params (#19351)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-26 16:48:53 +08:00
zhagnlu
59f8e96522
Change compactor policy from loading all binlogs to loading serially (#13716) (#19280)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-09-25 20:12:52 +08:00
Bingyi Sun
29d2c0bd7b
Move task merge cap to config file (#19387)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-09-24 17:56:51 +08:00
xige-16
1cd6e80c8a
Increase timeout of activate task (#19330)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-22 14:14:52 +08:00
xige-16
428840178c
Support diskann index for vector field (#19093)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-21 20:16:51 +08:00
Jiquan Long
5141e05c47
Add policy to seal long time idle segment (#19222)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-09-20 20:54:50 +08:00
bigsheeper
5143e2a75e
Disable quota and limits (#19242)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-20 09:34:50 +08:00
bigsheeper
5cdd6ca251
Add quotaCenter, rateLimiter and rateCollector (#18636)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-16 09:56:47 +08:00
Bingyi Sun
626854cf0c
Refactor QueryCoord (#18836)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-15 18:48:32 +08:00
bigsheeper
5eb6127fc5
Set max read concurrency ratio to improve concurrent read performance (#18911)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-08-30 19:50:57 +08:00
Xiaofan
8e22d03cf3
Support trigger compaction everyday to eliminiate deleted data in rocksmq (#18752)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-08-29 10:56:55 +08:00
cai.zhang
c924f73105
Refactor for IndexCoord to support cloud (#18643)
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
wgcn
d443b5420a
add user special kafka config (#18742)
Signed-off-by: wgcn <1026688210@qq.com>

Signed-off-by: wgcn <1026688210@qq.com>
Co-authored-by: wanggang11335 <wanggang11335@autohome.com.cn>
2022-08-25 11:02:53 +08:00
Xiaofan
8fe3895485
change retention time to 1 day (#18749)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-08-25 10:42:53 +08:00
SimFG
ce434b496e
Implement the mysql metastore of the rbac (#18704)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-23 10:26:53 +08:00
codeman
6c3dbf0a07
Support meta storing on database (#17236) (#18541)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-08-11 12:12:38 +08:00
bigsheeper
cef8b1e7cc
Enable jemalloc (#18349)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-20 22:22:31 +08:00
Jiquan Long
70246f82fd
Reduce chunk row size (#18320)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-07-19 13:50:28 +08:00
congqixia
f7bd054adb
Set EnableGarbageCollection to true by default (#18305)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-18 16:34:28 +08:00
cai.zhang
0ad15a22c8
IndexCoord handle events correctly (#17878)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-07 14:44:21 +08:00
zhenshan.cao
1647bebaf4
Count growing segment number to estimate cpu usage (#17820) (#18027)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-04 15:10:20 +08:00
jaime
762e3b78a6
support configurable sasl mechanisms for kafka (#17542)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-06-29 10:32:17 +08:00
shaoyue
96b97f7f58
RESTful api use same port as metrics & healthcheck (#17732)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-06-24 15:34:14 +08:00
SimFG
f4c6a6734e
Fix rpc retry policy not taking effect (#17673)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-06-22 22:22:13 +08:00
Xiaofan
d70f483806
Support rocksmq dynamic lru cache size (#17626)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-20 10:56:12 +08:00
bigsheeper
92d06b2e30
Purge memory by the memory state and try to purge after each search (#17565)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-17 17:46:10 +08:00
Xiaofan
1f6fbf91b2
Fix pulsar unsubsribe issue (#17562)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-16 17:28:11 +08:00
Ten Thousand Leaves
65ab847815
Add segment reference lock for bulk load (#17457)
So we can support DataQueryable and DataIndexed states

issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-15 12:20:10 +08:00
Bingyi Sun
b3f0bbadff
Adjust keepalive timeout to 20s as default (#17510)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-06-13 11:54:08 +08:00
yah01
a38790f586
Predict the loading memory usage more accurate (#17451)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-09 18:16:07 +08:00
shaoyue
76eaa3fc50
Support access s3 through AWS IAM role (#17292)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-06-02 19:42:03 +08:00
SimFG
0986c29d7f
proxy tls support one-way authentication (#17348)
Signed-off-by: xiyichan <2863768433@qq.com>

Co-authored-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>

Co-authored-by: xiyichan <2863768433@qq.com>
2022-06-02 18:56:04 +08:00
jaime
5698bf4236
Add sasl configurations for kafka (#17323)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-06-02 12:12:03 +08:00
Jiquan Long
6bd786bf50
Reduce default seal propotion (#17287)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-31 12:02:04 +08:00
congqixia
c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-30 19:50:04 +08:00
zhenshan.cao
2512e668f7
Update timestmap according to gracefultime for Bounded Consistency (#17171)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 12:05:59 +08:00
zhenshan.cao
08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
zhenshan.cao
246b22b5ea
Configure indexSliceSize to 16MB (#17080)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-23 15:01:58 +08:00
ownbylichaobao
2f8a7e7793
Support tls/ssl connection to etcd (#7703) (#17012)
Signed-off-by: lichaobao <1527563274@qq.com>
2022-05-20 12:29:19 +08:00
Jiquan Long
99b4042f8c
Disable segment small index (#16935)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-18 15:27:56 +08:00
exia
4ee78eb5ad
proxy support tls (#16706)
Signed-off-by: xiyichan <2863768433@qq.com>
2022-05-10 20:05:53 +08:00