Commit Graph

239 Commits

Author SHA1 Message Date
cai.zhang
b0c9f25352
Sort out the log of IndexCoord (#16810)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-09 12:05:52 +08:00
Cai Yudong
6a62ff18bf
Support easylogging config for segcore and knowhere (#16751)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-05-03 08:39:49 +08:00
xige-16
515d0369de
Support string type in segcore (#16546)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
2022-04-29 13:35:49 +08:00
zhenshan.cao
0a953948af
Simplify monitoring metrics (#16687)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-27 23:03:47 +08:00
Xiaofan
b657b10eb5
Remove useless log (#16624)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-27 10:01:47 +08:00
Xiaofan
89b4a34892
Fix Data race in NodeID (#16603)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-24 22:03:44 +08:00
Ji Bin
a5ac637ba5
Adaptation windows for Knowhere 1.1.3 (#16444)
fixes #16443

Signed-off-by: Ji Bin <matrixji@live.com>
2022-04-13 14:39:34 +08:00
Cai Yudong
71cd7ba67a
Add configuration common.indexSliceSize (#16438)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-04-08 20:29:33 +08:00
godchen
bb7a0766fe
Add dependency factory (#16204)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-07 22:05:32 +08:00
cai.zhang
e73139deeb
Compressed IndexNode log (#16365)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 11:33:31 +08:00
cai.zhang
04076393c5
Fix log error (#16363)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 10:13:32 +08:00
Jiquan Long
f8d9bc919d
Unify interface of vector index & scalar index. (#15959)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-21 14:23:24 +08:00
godchen
78557ca6d7
Replace minio kv with minio chunk manager (#15936)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-17 18:03:23 +08:00
Ji Bin
3cd28420f1
Support compile under windows (#15786)
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics (#16030)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
cai.zhang
2512dff261
Fix misuse of timerecord (#15868)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-04 15:45:56 +08:00
Cai Yudong
3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 11:17:56 +08:00
cai.zhang
09a27d1620
Add prometheus metrics for indexnode (#15640)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-02 20:49:56 +08:00
zhenshan.cao
f1aec49a74
Fixbug: building index return nil by mistake when error not occur (#15809)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-02 09:41:56 +08:00
zhenshan.cao
142848fcc3
Abandon using protobuf to pass binaryset parameter (#15626)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-02-18 18:39:50 +08:00
jaime
307a8ce535
Support compile and run on Mac (#15491)
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
2022-02-09 14:27:46 +08:00
Cai Yudong
1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-08 20:57:47 +08:00
Cai Yudong
f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-07 10:09:45 +08:00
zhenshan.cao
3f48a40ab0
Load index meta in every retry (#15376)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-25 13:28:05 +08:00
zhenshan.cao
6bc2efe429
Fixbug: IndexNode should panic when save meta failed to MetaKV (#15347)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-24 17:18:46 +08:00
zhenshan.cao
871e093dc8
Call release memory in case error occured (#15237)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-17 14:43:38 +08:00
Jiquan Long
2819f6cc6f
Release memory to OS after index is built (#15196)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-13 20:27:34 +08:00
cai.zhang
89c9073ab3
[skip e2e]Add comment for msgIndexNodeIsUnhealthy (#15142)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-12 11:11:36 +08:00
cai.zhang
4daf0fb6b5
[skip e2e]Add comment for errIndexNodeIsUnhealthy (#15141)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-12 11:09:36 +08:00
Cai Yudong
951249c6c1
Use KnowhereConfig for indexnode and querynode (#15112)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-11 09:35:34 +08:00
zhenshan.cao
1aafb56694
Clarify the index building process (#15044)
Reduce peak memory usage

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-10 20:51:34 +08:00
Cai Yudong
b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-10 17:29:34 +08:00
congqixia
fc256e7924
Index parallel respect CPU quota (#14991)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-07 17:31:22 +08:00
Cai Yudong
f8f61fb116
Use BaseParams etcd configuration for all components (#15012)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-07 16:15:26 +08:00
Jiquan Long
32eed4f025
Check leakage of index CGO object (#14630)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-06 23:28:15 +08:00
cai.zhang
2327d53915
[skip e2e]Add comment for setID (#14841)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-05 13:07:25 +08:00
cai.zhang
47b89ba84a
[skip e2e]Add comment for checkIndexMeta (#14842)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-05 13:05:25 +08:00
shaoyue
fc415c0fb5
[skip e2e] Add comment for indexnode.Params (#14721)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-01-04 10:45:31 +08:00
cai.zhang
d4c325adbb
[skip e2e]Reorder head files for indexnode/task.go (#14661)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-31 16:11:24 +08:00
cai.zhang
f1c3f465a5
[skip e2e]Reorder head files for indexnode/metrics_info.go (#14660)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-31 16:09:18 +08:00
cai.zhang
f96cd84efb
[skip e2e]Reorder head files for indexnode/indexnode_test.go (#14609)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-30 17:02:50 +08:00
cai.zhang
911fcf83a2
[skip e2e]Reorder head files for indexnode/indexnode_mock_test.go (#14608)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-30 17:01:07 +08:00
cai.zhang
4297ff9bd0
[skip e2e]Reorder head files for indexnode/error_test.go (#14607)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-30 16:36:24 +08:00
Xiaofan
69087ff8dd
Support embedded etcd (#14333)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 14:35:21 +08:00
Xiaofan
1408926cd9
Remove logs without enough info (#14440)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 10:06:47 +08:00
edward.zeng
9977bea982
[skip e2e] Refine indexnode task scheduler comment (#14450)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-28 19:50:25 +08:00
xige-16
a8cb995216
Improve log for build index in indexNode (#14051)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-23 22:00:16 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components (#13885)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
JackLCL
2c0d9397fb
[skip e2e]Fix error log output format in task.go (#14028)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-23 14:28:25 +08:00
cai.zhang
1dab9ccc7f
[skip e2e]Add commont for initMinIOAccessKeyID in indexnode (#13975)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-22 17:11:35 +08:00