Commit Graph

323 Commits

Author SHA1 Message Date
MrPresent-Han
5634ba777d
add new threadpool with various priority to avoid deadlock(#25781) (#26028)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-03 09:31:07 +08:00
zhagnlu
833674c1cb
add glog configurable function and redirect aws log to segcore log (#25664)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-07-27 19:49:02 +08:00
congqixia
3c503afe7c
Use typeutil.ConcurrentMap instead of sync.Map (#25846)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-24 10:23:01 +08:00
cqy123456
0f3350ded4
set knowhere thread pool size in index node (#25804)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-07-21 18:50:59 +08:00
Enwei Jiao
66fdc71479
Refactor logs in DataCoord & DataNode (#25574)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-14 15:56:31 +08:00
congqixia
28200acb67
Fix indexnode memory leakage when update index fails (#25460)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-11 11:18:28 +08:00
xige-16
33c2012675
Add more metrics (#25081)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-26 17:52:44 +08:00
xige-16
04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yah01
ebd0279d3f
Check error by Error() and NoError() for better report message (#24736)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
yiwangdr
37c02c9927
clean up meta interface (#24729)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-06-08 14:18:35 +08:00
aoiasd
c84bdcea49
merge stats log when segment flushing or compacting (#23570)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-05-29 10:21:28 +08:00
xige-16
fe18109aab
Clean tmp disk data when start milvus (#24404)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-05-26 15:07:26 +08:00
smellthemoon
0b72cf2c2e
log out error (#24260)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-19 18:25:24 +08:00
congqixia
73a181d226
Fix get vector it timeout and improve some string const usage (#24141)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-16 17:41:22 +08:00
cai.zhang
43a9e175a3
Exit component process when session key is deleted (#21658) (#22164)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-04-12 20:12:28 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
congqixia
127867b873
Add ratedgroup for some info/warning log (#23095)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-31 15:22:23 +08:00
zhenshan.cao
1287ca699a
Refine usage of TimeRecorder.Record (#23142)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-03-30 18:56:22 +08:00
Cai Yudong
8aebc6f3b7
Remove faiss GPU index support (#22966)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-03-24 17:53:58 +08:00
congqixia
8f847884c2
Fix misc copylocks and remove exclude rule (#22962)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-24 11:17:58 +08:00
Cai Yudong
ab3cbdfc61
Partial change to prepare for GPU index type support (#22591)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-03-14 23:21:56 +08:00
congqixia
9e17e2660e
Fix lifetime race in indexnode (#22645)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-09 16:19:52 +08:00
cai.zhang
5cae49e28d
Store index files in slices to reduce memory peaks (#22369) (#22455)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-03-01 19:43:47 +08:00
yah01
7478e44911
Support using mmap to load data (#22052)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-01 18:07:49 +08:00
Enwei Jiao
697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
zhenshan.cao
e768437681
Correct usage of Timer and Ticker (#22228)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-02-23 18:59:45 +08:00
congqixia
ec282b1958
Apply lifetime control for indexnode (#21892)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-31 18:37:50 +08:00
Xiaofan
949d5d078f
Fix memory calculation in dataCodec (#21800)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-28 11:09:52 +08:00
congqixia
5986106037
Make paramtable init only once (#21782)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-19 14:53:44 +08:00
Enwei Jiao
90d9e165d4
Fix some configs not shown (#21653)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-13 15:31:41 +08:00
wayblink
6a722396bd
Integration test framework (#21283)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-01-12 19:49:40 +08:00
Enwei Jiao
fb42466c65
Use opentelemetry (#21509)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-12 16:09:39 +08:00
cai.zhang
aa203acfb3
Low IndexCoord weight (#21548)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-06 14:21:37 +08:00
cai.zhang
e5f408dceb
Merge IndexCoord and DataCoord (#21267)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-04 19:37:36 +08:00
yah01
7b39873ae0
limit the frequency of GetMetrics() log (#21514)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-04 17:39:35 +08:00
SimFG
4df91ddf60
Support the graceful stop for the indexnode (#21021)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-12-16 10:03:27 +08:00
aoiasd
de0ab9e2cf
Refactor showConfigurations to allow return global config rather than only return config of this component (#21063)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-12-09 14:31:21 +08:00
Enwei Jiao
89b810a4db
Refactor all params into ParamItem (#20987)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-07 18:01:19 +08:00
liliu-z
aecf2e4f4b
Upgrade knowhere to 1.3.4 (#20914)
Signed-off-by: Li Liu <li.liu@zilliz.com>

Signed-off-by: Li Liu <li.liu@zilliz.com>
2022-12-02 00:33:21 +08:00
yah01
882ba6240b
Check whether CodecIndex is nil to avoid panic (#20816)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-25 10:45:12 +08:00
Enwei Jiao
c05b9ad539
Add event dispatcher for config (#20393)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 18:59:09 +08:00
Enwei Jiao
19524a5344
Fix nodeID mismatch at standalone mode (#20648)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 17:15:08 +08:00
cai.zhang
889eb02172
Add goroutine pool for creating index (#20391)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-11-08 19:41:02 +08:00
Enwei Jiao
7e56e7a976
Fix datarace at Setlogger (#20376)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-08 14:33:03 +08:00
cai.zhang
8ea8f91334
Make index log better (#20327)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-11-07 14:23:02 +08:00
Enwei Jiao
956c5e1b9d
Make Params singleton (#20088)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-04 14:25:38 +08:00
zhagnlu
39212bc04c
Fix get cpu num for segcore (#20120) (#20307)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-11-03 19:29:36 +08:00
Jiquan Long
e7103310b8
Fix log when building index (#20302)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-11-03 18:19:35 +08:00
zhagnlu
cb2591d1fe
Parallel operations of disk-ann file manager (#20120) (#20122)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-11-03 14:39:40 +08:00