Commit Graph

228 Commits

Author SHA1 Message Date
Jiquan Long
f8d9bc919d
Unify interface of vector index & scalar index. (#15959)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-21 14:23:24 +08:00
godchen
78557ca6d7
Replace minio kv with minio chunk manager (#15936)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-17 18:03:23 +08:00
Ji Bin
3cd28420f1
Support compile under windows (#15786)
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics (#16030)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
cai.zhang
2512dff261
Fix misuse of timerecord (#15868)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-04 15:45:56 +08:00
Cai Yudong
3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 11:17:56 +08:00
cai.zhang
09a27d1620
Add prometheus metrics for indexnode (#15640)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-02 20:49:56 +08:00
zhenshan.cao
f1aec49a74
Fixbug: building index return nil by mistake when error not occur (#15809)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-02 09:41:56 +08:00
zhenshan.cao
142848fcc3
Abandon using protobuf to pass binaryset parameter (#15626)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-02-18 18:39:50 +08:00
jaime
307a8ce535
Support compile and run on Mac (#15491)
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
2022-02-09 14:27:46 +08:00
Cai Yudong
1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-08 20:57:47 +08:00
Cai Yudong
f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-07 10:09:45 +08:00
zhenshan.cao
3f48a40ab0
Load index meta in every retry (#15376)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-25 13:28:05 +08:00
zhenshan.cao
6bc2efe429
Fixbug: IndexNode should panic when save meta failed to MetaKV (#15347)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-24 17:18:46 +08:00
zhenshan.cao
871e093dc8
Call release memory in case error occured (#15237)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-17 14:43:38 +08:00
Jiquan Long
2819f6cc6f
Release memory to OS after index is built (#15196)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-13 20:27:34 +08:00
cai.zhang
89c9073ab3
[skip e2e]Add comment for msgIndexNodeIsUnhealthy (#15142)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-12 11:11:36 +08:00
cai.zhang
4daf0fb6b5
[skip e2e]Add comment for errIndexNodeIsUnhealthy (#15141)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-12 11:09:36 +08:00
Cai Yudong
951249c6c1
Use KnowhereConfig for indexnode and querynode (#15112)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-11 09:35:34 +08:00
zhenshan.cao
1aafb56694
Clarify the index building process (#15044)
Reduce peak memory usage

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-10 20:51:34 +08:00
Cai Yudong
b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-10 17:29:34 +08:00
congqixia
fc256e7924
Index parallel respect CPU quota (#14991)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-07 17:31:22 +08:00
Cai Yudong
f8f61fb116
Use BaseParams etcd configuration for all components (#15012)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-07 16:15:26 +08:00
Jiquan Long
32eed4f025
Check leakage of index CGO object (#14630)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-06 23:28:15 +08:00
cai.zhang
2327d53915
[skip e2e]Add comment for setID (#14841)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-05 13:07:25 +08:00
cai.zhang
47b89ba84a
[skip e2e]Add comment for checkIndexMeta (#14842)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-05 13:05:25 +08:00
shaoyue
fc415c0fb5
[skip e2e] Add comment for indexnode.Params (#14721)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-01-04 10:45:31 +08:00
cai.zhang
d4c325adbb
[skip e2e]Reorder head files for indexnode/task.go (#14661)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-31 16:11:24 +08:00
cai.zhang
f1c3f465a5
[skip e2e]Reorder head files for indexnode/metrics_info.go (#14660)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-31 16:09:18 +08:00
cai.zhang
f96cd84efb
[skip e2e]Reorder head files for indexnode/indexnode_test.go (#14609)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-30 17:02:50 +08:00
cai.zhang
911fcf83a2
[skip e2e]Reorder head files for indexnode/indexnode_mock_test.go (#14608)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-30 17:01:07 +08:00
cai.zhang
4297ff9bd0
[skip e2e]Reorder head files for indexnode/error_test.go (#14607)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-30 16:36:24 +08:00
Xiaofan
69087ff8dd
Support embedded etcd (#14333)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 14:35:21 +08:00
Xiaofan
1408926cd9
Remove logs without enough info (#14440)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 10:06:47 +08:00
edward.zeng
9977bea982
[skip e2e] Refine indexnode task scheduler comment (#14450)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-28 19:50:25 +08:00
xige-16
a8cb995216
Improve log for build index in indexNode (#14051)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-23 22:00:16 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components (#13885)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
JackLCL
2c0d9397fb
[skip e2e]Fix error log output format in task.go (#14028)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-23 14:28:25 +08:00
cai.zhang
1dab9ccc7f
[skip e2e]Add commont for initMinIOAccessKeyID in indexnode (#13975)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-22 17:11:35 +08:00
cai.zhang
595692c350
[skip e2e]Add commont for initMinIOAddress in indexnode (#13973)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-22 17:03:43 +08:00
cai.zhang
9f23fc7f2a
Register the service when the component state is healthy (#13248)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-15 11:47:10 +08:00
JackLCL
7cb35e4172
[skip ci]Fix error log output format in task.go (#13102)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-09 19:45:28 +08:00
cai.zhang
342200ce13
Estimate the memory size of the index before building the index (#12973)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-09 14:19:40 +08:00
yanliang567
694e3f72a1
[skip ci]Refine etcd name in comments (#12798)
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2021-12-06 18:59:55 +08:00
congqixia
38956215aa
Update indexnode state to unhealthy before stopping (#12744)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-06 10:39:36 +08:00
godchen
5e3e3b3b41
Remove grpc returned error (#12537)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-12-01 22:17:46 +08:00
jaime
90a1d016a0
Refine return value (#11730)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-11-30 19:32:17 +08:00
dragondriver
b6d0150dc5
Fix health check when disconnect to etcd (#12292)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-26 11:39:16 +08:00
dragondriver
6c4c0ef6b5
Add more deploy metrics (#12248)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-24 15:43:15 +08:00
godchen
9d5bcd3e3a
Close event and binlog reader (#12173)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-22 17:27:14 +08:00