Commit Graph

296 Commits

Author SHA1 Message Date
congqixia
ec282b1958
Apply lifetime control for indexnode (#21892)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-31 18:37:50 +08:00
Xiaofan
949d5d078f
Fix memory calculation in dataCodec (#21800)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-28 11:09:52 +08:00
congqixia
5986106037
Make paramtable init only once (#21782)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-19 14:53:44 +08:00
Enwei Jiao
90d9e165d4
Fix some configs not shown (#21653)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-13 15:31:41 +08:00
wayblink
6a722396bd
Integration test framework (#21283)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-01-12 19:49:40 +08:00
Enwei Jiao
fb42466c65
Use opentelemetry (#21509)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-12 16:09:39 +08:00
cai.zhang
aa203acfb3
Low IndexCoord weight (#21548)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-06 14:21:37 +08:00
cai.zhang
e5f408dceb
Merge IndexCoord and DataCoord (#21267)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-04 19:37:36 +08:00
yah01
7b39873ae0
limit the frequency of GetMetrics() log (#21514)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-04 17:39:35 +08:00
SimFG
4df91ddf60
Support the graceful stop for the indexnode (#21021)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-12-16 10:03:27 +08:00
aoiasd
de0ab9e2cf
Refactor showConfigurations to allow return global config rather than only return config of this component (#21063)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-12-09 14:31:21 +08:00
Enwei Jiao
89b810a4db
Refactor all params into ParamItem (#20987)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-07 18:01:19 +08:00
liliu-z
aecf2e4f4b
Upgrade knowhere to 1.3.4 (#20914)
Signed-off-by: Li Liu <li.liu@zilliz.com>

Signed-off-by: Li Liu <li.liu@zilliz.com>
2022-12-02 00:33:21 +08:00
yah01
882ba6240b
Check whether CodecIndex is nil to avoid panic (#20816)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-25 10:45:12 +08:00
Enwei Jiao
c05b9ad539
Add event dispatcher for config (#20393)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 18:59:09 +08:00
Enwei Jiao
19524a5344
Fix nodeID mismatch at standalone mode (#20648)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 17:15:08 +08:00
cai.zhang
889eb02172
Add goroutine pool for creating index (#20391)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-11-08 19:41:02 +08:00
Enwei Jiao
7e56e7a976
Fix datarace at Setlogger (#20376)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-08 14:33:03 +08:00
cai.zhang
8ea8f91334
Make index log better (#20327)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-11-07 14:23:02 +08:00
Enwei Jiao
956c5e1b9d
Make Params singleton (#20088)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-04 14:25:38 +08:00
zhagnlu
39212bc04c
Fix get cpu num for segcore (#20120) (#20307)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-11-03 19:29:36 +08:00
Jiquan Long
e7103310b8
Fix log when building index (#20302)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-11-03 18:19:35 +08:00
zhagnlu
cb2591d1fe
Parallel operations of disk-ann file manager (#20120) (#20122)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-11-03 14:39:40 +08:00
xige-16
5bb2626466
Set diskUsageRatio to 4.0 when build disk index (#20066)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-26 16:11:31 +08:00
cai.zhang
e7d58b4fb7
Set index state as retry when error occurred (#19978)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-25 11:09:30 +08:00
jaime
b79687687d
Replace segment index file path with index file id within segment index meta (#19866)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-19 16:55:27 +08:00
xige-16
158787811e
Move assemble/disassemble func to core (#19420)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-16 21:17:25 +08:00
SimFG
a55f739608
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
xige-16
a1db9038fb
Move disk index params to config file (#19714)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-14 17:51:24 +08:00
xige-16
8c9c1672ae
Assign different storage config for indexes (#19517)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-14 14:45:23 +08:00
wayblink
85e04d845c
Move GetComponentStates to milvus.proto, make it an external interface (#19594)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-10-10 15:55:22 +08:00
zhenshan.cao
0ac4bc32a5
Add logic of autoindex (#19570)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-10-08 15:38:58 +08:00
congqixia
838a633584
Add ctx parameter for ChunkManager methods (#19546)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-29 16:18:56 +08:00
cai.zhang
c58a8d8fd6
Support bind IndexNode mode (#19247)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-27 22:38:55 +08:00
cai.zhang
2442335aa2
Fix indexnode ut bug (#19468)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-27 14:56:52 +08:00
xige-16
3fb8e80147
Rename serializeDiskIndex to getIndexFileInfo (#19416)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-27 10:32:53 +08:00
cai.zhang
9e18645363
Optimize IndexCoord scheduler loop (#19375)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-24 13:40:52 +08:00
Xiaofan
928a213e31
Fix Chunk Manager Path mis use (#19141)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-09-23 14:40:51 +08:00
Xiaofan
901d3fb367
Add more log when creating index, Finished/Failed index will not be counted in total index tasks (#19150)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-09-23 09:40:52 +08:00
xige-16
428840178c
Support diskann index for vector field (#19093)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-21 20:16:51 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
cai.zhang
e65ba6331e
Only store segmentID for flushed segments event (#19075)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-09 15:52:35 +08:00
cai.zhang
d6b0fdc944
Fill index size and fail reason when query index job (#19026)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-06 17:19:11 +08:00
congqixia
c91bb0b015
Unify rootPath in configs and ChunkManager (#18808)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-25 19:32:53 +08:00
cai.zhang
c924f73105
Refactor for IndexCoord to support cloud (#18643)
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
aoiasd
8a70f2badb
Add ShowConfigurations for all remaining session (#18594)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-08-12 13:20:39 +08:00
congqixia
be6ca6358d
Use serialized size before compression in log meta (#18336)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-19 21:30:30 +08:00
cai.zhang
93d9dfad1a
Ignore task when IndexNode load empty index meta (#18266)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-14 15:08:28 +08:00
cai.zhang
a47a414dce
Fix bug for IndexNode pod kill test (#18177)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-13 16:16:25 +08:00
cai.zhang
90faf4a212
Fix bug for IndexCoord recycles index files (#18168)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-08 16:54:22 +08:00