Commit Graph

60 Commits

Author SHA1 Message Date
Ten Thousand Leaves
b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-09-26 18:06:54 +08:00
zhagnlu
59f8e96522
Change compactor policy from loading all binlogs to loading serially (#13716) (#19280)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-09-25 20:12:52 +08:00
Bingyi Sun
29d2c0bd7b
Move task merge cap to config file (#19387)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-09-24 17:56:51 +08:00
SimFG
68a257458b
Implement the hook interceptor (#19294)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-23 09:50:52 +08:00
xige-16
1cd6e80c8a
Increase timeout of activate task (#19330)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-22 14:14:52 +08:00
xige-16
428840178c
Support diskann index for vector field (#19093)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-21 20:16:51 +08:00
Jiquan Long
5141e05c47
Add policy to seal long time idle segment (#19222)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-09-20 20:54:50 +08:00
bigsheeper
5cdd6ca251
Add quotaCenter, rateLimiter and rateCollector (#18636)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-16 09:56:47 +08:00
Bingyi Sun
626854cf0c
Refactor QueryCoord (#18836)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-15 18:48:32 +08:00
bigsheeper
5eb6127fc5
Set max read concurrency ratio to improve concurrent read performance (#18911)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-08-30 19:50:57 +08:00
congqixia
c91bb0b015
Unify rootPath in configs and ChunkManager (#18808)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-25 19:32:53 +08:00
cai.zhang
c924f73105
Refactor for IndexCoord to support cloud (#18643)
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
Xiaofan
8fe3895485
change retention time to 1 day (#18749)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-08-25 10:42:53 +08:00
Enwei Jiao
d9c7519107
refactor compaction concurency logic (#18660)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-23 15:50:52 +08:00
codeman
6c3dbf0a07
Support meta storing on database (#17236) (#18541)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-08-11 12:12:38 +08:00
bigsheeper
002f509808
Remove deprecated cacheSize param (#18566)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-08-09 15:54:36 +08:00
SimFG
ff0200210a
Support Role-Based Access Control (#18425)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-04 11:04:34 +08:00
Enwei Jiao
283f5731d2
config from etcd (#18421)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-01 10:04:33 +08:00
bigsheeper
cef8b1e7cc
Enable jemalloc (#18349)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-20 22:22:31 +08:00
Jiquan Long
70246f82fd
Reduce chunk row size (#18320)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-07-19 13:50:28 +08:00
congqixia
f7bd054adb
Set EnableGarbageCollection to true by default (#18305)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-18 16:34:28 +08:00
Xiaofan
c97563590c
fix missing handling channels while a node down (#18250)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-07-18 13:06:28 +08:00
bigsheeper
1da970af18
Remove useless statsService in QueryNode (#17395)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-11 15:12:25 +08:00
cai.zhang
0ad15a22c8
IndexCoord handle events correctly (#17878)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-07 14:44:21 +08:00
congqixia
8f1ba6a0bb
Concurrent load segment stats log when initialize dataSyncService (#18125)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-07 14:32:21 +08:00
zhenshan.cao
1647bebaf4
Count growing segment number to estimate cpu usage (#17820) (#18027)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-04 15:10:20 +08:00
bigsheeper
92d06b2e30
Purge memory by the memory state and try to purge after each search (#17565)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-17 17:46:10 +08:00
Xiaofan
54aeb077e9
Refine compaction selection poliy (#17486)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-15 23:14:10 +08:00
Ten Thousand Leaves
65ab847815
Add segment reference lock for bulk load (#17457)
So we can support DataQueryable and DataIndexed states

issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-15 12:20:10 +08:00
yah01
a38790f586
Predict the loading memory usage more accurate (#17451)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-09 18:16:07 +08:00
Xiaofan
a9ec1f8864
Fix compatibility of upgrade (#17406)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-07 14:52:06 +08:00
XuanYang-cn
c70af73005
Fix datacoord set wrong state for node registering (#17376)
Fix datacoord datarace

See also: #17335

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-06 15:10:05 +08:00
Jiquan Long
6bd786bf50
Reduce default seal propotion (#17287)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-31 12:02:04 +08:00
congqixia
c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-30 19:50:04 +08:00
zhenshan.cao
2512e668f7
Update timestmap according to gracefultime for Bounded Consistency (#17171)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 12:05:59 +08:00
zhenshan.cao
cce533c246
Fix unitest fail, change default index slice to 16 (#17162)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-23 17:44:25 +08:00
zhenshan.cao
08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
Enwei Jiao
67ce206cf0
make SegmentMaxLifetime configurable (#16783)
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
2022-05-06 23:41:51 +08:00
Ten Thousand Leaves
1acd256481
Add DataQueryable and DataIndexed states for bulk load tasks (#16725)
issue: #16607

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-05 21:17:50 +08:00
Xiaofan
333b439a6c
Support config grpc log level (#16618)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-05 10:35:50 +08:00
shaoyue
fa97f86cda
Clean up not used id_cache related code (#16655)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-05-05 09:59:51 +08:00
Xiaofan
337ad53613
Support config small index nprobe and nlist (#16731)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-05 09:49:50 +08:00
Xiaofan
ed51ffd50f
Fix Compact Entity ttl param (#16665)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-28 11:51:47 +08:00
Xiaofan
89b4a34892
Fix Data race in NodeID (#16603)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-24 22:03:44 +08:00
codeman
3a1b2cedd2
create default root user for authentication (#16545) (#16549)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-04-20 13:03:40 +08:00
Xiaofan
f367249b9e
Fix storage type config key (#16542)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-19 17:47:39 +08:00
codeman
f5e1ded854
fix #16504 (#16523)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-04-19 16:35:39 +08:00
Xiaofan
fce8f6cfab
Remove Timetick channel (#16449)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-17 23:47:36 +08:00
jaime
ed3c29eae7
Support Kafka (#16149)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-12 19:47:33 +08:00
codeman
4e12271d79
Support login with username and password (#15656) (#16341)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-04-11 19:49:34 +08:00