Commit Graph

91 Commits

Author SHA1 Message Date
cai.zhang
0ad15a22c8
IndexCoord handle events correctly (#17878)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-07 14:44:21 +08:00
cai.zhang
01fc411566
Handle etcd compacted error (#17886)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-30 18:54:19 +08:00
cai.zhang
899a9bf86d
IndexNode still writes finished state to ETCD when task was deleted (#17761)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-28 12:08:17 +08:00
cai.zhang
ef0fc37ad7
Fixed the bug that IndexCoord lost some meta information (#17818)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-27 21:52:17 +08:00
cai.zhang
e8f53af749
When the task is canceled, reduce the load of node (#17735)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-23 19:22:15 +08:00
cai.zhang
1fbdafc943
IndexCoord does not assign task to IndexNode prematurely (#17717)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-23 10:40:13 +08:00
cai.zhang
141a3bbe00
Mark deleted before drop index (#17490)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-17 18:08:12 +08:00
cai.zhang
ea5041aec2
Acquiring the segment reference lock on task level (#17544)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-15 21:38:10 +08:00
Xiaofan
af994b5e1c
Fix etcd watch error (#17364)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-06 16:26:06 +08:00
cai.zhang
bcf3b7426a
Add distributed lock for segment refgerence (#16782)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
cai.zhang
b37b87eb97
The GetIndexFilePaths interface judges whether the index is complete (#17052)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-17 20:41:56 +08:00
cai.zhang
b0c9f25352
Sort out the log of IndexCoord (#16810)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-09 12:05:52 +08:00
cai.zhang
2be46a01c0
Reduce the number of retries and add error log (#16754)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-05-05 09:31:51 +08:00
godchen
bb7a0766fe
Add dependency factory (#16204)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-07 22:05:32 +08:00
cai.zhang
4267dc2efc
Handle error and oragnize log (#16367)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 14:33:30 +08:00
cai.zhang
f9c6b3980f
Compressed log (#16354)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 10:11:36 +08:00
cai.zhang
e2e345079f
Add indexMeta version for IndexCoord log (#16266)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-29 21:03:27 +08:00
cai.zhang
c8688a6275
Organize log for GetIndexFilePaths (#16230)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-28 21:27:27 +08:00
godchen
78557ca6d7
Replace minio kv with minio chunk manager (#15936)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-17 18:03:23 +08:00
Ji Bin
3cd28420f1
Support compile under windows (#15786)
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
cai.zhang
b435c422c1
Add prometheus metrics for IndexCoord (#15638)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-02-21 17:15:51 +08:00
congqixia
c7f55c2e34
Make SessionWatch keep watch even no Rewatch func when find ErrCompacted (#15497)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-15 15:07:48 +08:00
Cai Yudong
1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-08 20:57:47 +08:00
Cai Yudong
f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-07 10:09:45 +08:00
Cai Yudong
b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-10 17:29:34 +08:00
Cai Yudong
f8f61fb116
Use BaseParams etcd configuration for all components (#15012)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-07 16:15:26 +08:00
Jiquan Long
4f71fff388
Fix panic when close IndexCoord (#14611)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-04 10:19:18 +08:00
Xiaofan
69087ff8dd
Support embedded etcd (#14333)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 14:35:21 +08:00
Xiaofan
1408926cd9
Remove logs without enough info (#14440)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 10:06:47 +08:00
cai.zhang
4d21874aab
[skip e2e]Reorder head files for indexcoord/indexcoord.go (#14296)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-27 17:20:06 +08:00
zhenshan.cao
3b7dad4486
Add node ctx mananger logic for IndexCoord (#13862)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-12-23 21:35:52 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components (#13885)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
cai.zhang
2ef2228ad0
Check component state before get index files (#13578)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-17 23:10:42 +08:00
cai.zhang
037b9b0d4e
[skip e2e]Format log (#13579)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-17 22:17:03 +08:00
cai.zhang
461130e933
[skip e2e]Add log for BuildIndex (#13577)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-17 22:15:08 +08:00
cai.zhang
6304717f81
Adjust the order of import packages for indexcoord.go (#13500)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-16 15:30:04 +08:00
cai.zhang
9f23fc7f2a
Register the service when the component state is healthy (#13248)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-15 11:47:10 +08:00
Xiangyu Wang
405b3cd932
Use the same lint rules with golint in revive (#13324)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-14 15:31:07 +08:00
cai.zhang
af46998a2a
The memory of indexnode must be 6 times the segment size (#13213)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-13 15:06:29 +08:00
cai.zhang
342200ce13
Estimate the memory size of the index before building the index (#12973)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-09 14:19:40 +08:00
cai.zhang
961e9379c6
Check the indexcoord state before get index state (#12452)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-09 11:43:09 +08:00
congqixia
de454956fe
Refactor session WatchSessions to allow rewatch when Rev Compacted (#12880)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-08 10:11:04 +08:00
cai.zhang
4175fde7bc
Check the indexcoord state before drop index (#12453)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-07 21:07:22 +08:00
cai.zhang
926f47e318
Add log for indexcoord (#12789)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-07 09:55:56 +08:00
yanliang567
b2402cd916
[skip ci]Update etcd name in comments (#12800)
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2021-12-06 19:01:49 +08:00
cai.zhang
c3e0299f06
[skip ci]Add comment for assignTask in indexcoord (#12788)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-06 18:29:54 +08:00
congqixia
19000e6af2
Update indexcoord state to unhealthy before stopping (#12723)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-06 10:37:40 +08:00
Xiaofan
0f5776e5fb
Index coord should not get all node from etcd each time (#12668)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-04 11:39:34 +08:00
cai.zhang
ed546d0a55
Check the indexcoord state before building index (#12451)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-02 23:11:35 +08:00
godchen
5e3e3b3b41
Remove grpc returned error (#12537)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-12-01 22:17:46 +08:00