cai.zhang
93d9dfad1a
Ignore task when IndexNode load empty index meta ( #18266 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-14 15:08:28 +08:00
cai.zhang
90faf4a212
Fix bug for IndexCoord recycles index files ( #18168 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-08 16:54:22 +08:00
cai.zhang
0ad15a22c8
IndexCoord handle events correctly ( #17878 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-07 14:44:21 +08:00
Xiaofan
a9b1d71a8f
change the etcd compareAndSwap interface ( #18068 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-07-06 13:54:21 +08:00
cai.zhang
899a9bf86d
IndexNode still writes finished state to ETCD when task was deleted ( #17761 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-28 12:08:17 +08:00
cai.zhang
3685ff9e0c
Mark indexing task to retry when minio occurs error ( #17153 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-23 15:47:58 +08:00
cai.zhang
b37b87eb97
The GetIndexFilePaths interface judges whether the index is complete ( #17052 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-17 20:41:56 +08:00
xige-16
515d0369de
Support string type in segcore ( #16546 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
2022-04-29 13:35:49 +08:00
zhenshan.cao
0a953948af
Simplify monitoring metrics ( #16687 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-27 23:03:47 +08:00
Xiaofan
b657b10eb5
Remove useless log ( #16624 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-27 10:01:47 +08:00
Xiaofan
89b4a34892
Fix Data race in NodeID ( #16603 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-24 22:03:44 +08:00
godchen
bb7a0766fe
Add dependency factory ( #16204 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-07 22:05:32 +08:00
cai.zhang
04076393c5
Fix log error ( #16363 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 10:13:32 +08:00
Jiquan Long
f8d9bc919d
Unify interface of vector index & scalar index. ( #15959 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-21 14:23:24 +08:00
godchen
78557ca6d7
Replace minio kv with minio chunk manager ( #15936 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-17 18:03:23 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics ( #16030 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
cai.zhang
2512dff261
Fix misuse of timerecord ( #15868 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-04 15:45:56 +08:00
cai.zhang
09a27d1620
Add prometheus metrics for indexnode ( #15640 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-02 20:49:56 +08:00
zhenshan.cao
f1aec49a74
Fixbug: building index return nil by mistake when error not occur ( #15809 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-02 09:41:56 +08:00
zhenshan.cao
3f48a40ab0
Load index meta in every retry ( #15376 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-25 13:28:05 +08:00
zhenshan.cao
6bc2efe429
Fixbug: IndexNode should panic when save meta failed to MetaKV ( #15347 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-24 17:18:46 +08:00
zhenshan.cao
871e093dc8
Call release memory in case error occured ( #15237 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-17 14:43:38 +08:00
Jiquan Long
2819f6cc6f
Release memory to OS after index is built ( #15196 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-13 20:27:34 +08:00
zhenshan.cao
1aafb56694
Clarify the index building process ( #15044 )
...
Reduce peak memory usage
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-10 20:51:34 +08:00
congqixia
fc256e7924
Index parallel respect CPU quota ( #14991 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-07 17:31:22 +08:00
cai.zhang
2327d53915
[skip e2e]Add comment for setID ( #14841 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-05 13:07:25 +08:00
cai.zhang
47b89ba84a
[skip e2e]Add comment for checkIndexMeta ( #14842 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-05 13:05:25 +08:00
cai.zhang
d4c325adbb
[skip e2e]Reorder head files for indexnode/task.go ( #14661 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-31 16:11:24 +08:00
xige-16
a8cb995216
Improve log for build index in indexNode ( #14051 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-23 22:00:16 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components ( #13885 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
JackLCL
2c0d9397fb
[skip e2e]Fix error log output format in task.go ( #14028 )
...
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-23 14:28:25 +08:00
JackLCL
7cb35e4172
[skip ci]Fix error log output format in task.go ( #13102 )
...
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-09 19:45:28 +08:00
cai.zhang
342200ce13
Estimate the memory size of the index before building the index ( #12973 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-09 14:19:40 +08:00
jaime
90a1d016a0
Refine return value ( #11730 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-11-30 19:32:17 +08:00
godchen
9d5bcd3e3a
Close event and binlog reader ( #12173 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-22 17:27:14 +08:00
Baiqiang Zhao
ba9977951a
Fix log in indexnode ( #11988 )
...
Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
2021-11-17 16:41:13 +08:00
XuanYang-cn
35e8779bd9
Rename IndexRootPath to IndexStorageRootPath ( #11236 )
...
IndexRootPath is index file's blob storage prefix.
But the name is confusing with the MetaRootPath, which is prefix of etcd.
This PR changes the IndexRootPath to IndexStorageRootPath to
elimilate the confusion.
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-05 11:45:00 +08:00
XuanYang-cn
a086042b45
[skip ci]Update in task ( #11032 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-01 22:51:41 +08:00
groot
d33e810aa1
[skip ci] Fix a typo ( #10155 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-10-19 16:24:35 +08:00
Xiaofan
91d8f85df7
Refine log in task.go ( #8992 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-13 23:24:32 +08:00
cai.zhang
7801357dd3
Retry index building task in case of internal error such as minio failure ( #9552 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-10-11 16:38:33 +08:00
cai.zhang
6b58c5930e
[skip ci]Fix golint for BaseTask in indexnode.go ( #9495 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-10-09 18:27:00 +08:00
cai.zhang
e368b51d3e
[skip ci]Fix golint for task executing in indexnode.go ( #9494 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-10-09 18:24:58 +08:00
cai.zhang
3b7b10c18e
[skip ci]Fix golint for IndexBuildTask in indexnode.go ( #9493 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-10-09 00:05:49 +08:00
zhenshan.cao
d1d5f9e409
Replace proto.MarshalTextString with proto.Marshal ( #8542 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-29 20:26:00 +08:00
dragondriver
cf8600077f
Refactor the index file format ( #8514 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-29 09:52:12 +08:00
cai.zhang
df30315144
Standard indexnode log printing ( #8587 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-26 21:22:04 +08:00
cai.zhang
0fa38d5d93
Add root path for index files in minio ( #8212 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-26 19:19:57 +08:00
cai.zhang
d51fec6678
Enhance log when create index failed ( #6871 )
...
* Enhance log when create index failed
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
* Fix bug for repeatedly drop index
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-29 14:47:22 +08:00
cai.zhang
e1b52b0583
Add unittest for IndexCoord and IndexNode ( #6698 )
...
* Add unittest for IndexCoord and IndexNode
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
* Fix verifiers
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-23 10:44:12 +08:00
cai.zhang
9aa56cd7b8
Add log for IndexCoord and IndexNode ( #6597 )
...
* Add log for IndexCoord and IndexNode
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
* Make verifier
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
* Delete unused script
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-22 11:40:11 +08:00
Cai Yudong
a7a9eeeffe
Add time record for build index and search process ( #6231 )
...
* add timerecord for build index process
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix build error
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add timerecord for query process
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-30 19:46:14 +08:00
godchen
99be4c09a5
change retry ( #5996 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-23 09:24:10 +08:00
zhenshan.cao
5c69c6902d
Refactor proxy channelTimeTick and fix search timeout ( #5760 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:36:36 +08:00
zhenshan.cao
bc46e4780d
Fixbug:index is not loaded in some cases ( #5633 )
...
* Fixbug:index is not created in some cases
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
* Delete unecessary print
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
* fix create index
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-06 09:41:35 +08:00
cai.zhang
a606ab9c67
Add power-off restart logic for IndexService and IndexNode ( #5395 )
...
Add power-off restart logic for IndexService and IndexNode
Signed-off-by: <xiaocai2333> <cai.zhang@zilliz.com>
2021-05-27 14:24:29 +00:00
Xiangyu Wang
82ccd4cec0
Rename module ( #4988 )
...
* Rename module
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
cai.zhang
8a037a404e
Add Copyright for indexnode and indexservice
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-04-19 13:50:12 +08:00
godchen
deba964590
Change name of kvBase
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-12 18:09:28 +08:00
dragondriver
41e1975611
Parallelize the processing of loading binlog and saving index files
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-03-26 11:19:02 +08:00
cai.zhang
012f5f360e
Add init log logic for index and proxy
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-03-12 19:47:37 +08:00
godchen
f3649f0419
Refactor interface and proto
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-03-12 14:22:09 +08:00
xige-16
09ae985daa
Fix wrong error code in master_service_test.go
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-10 22:06:22 +08:00
cai.zhang
60b97f9b65
Replace log of indexnode and indexservice
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-03-10 09:56:09 +08:00
quicksilver
455cc59dbb
Update publish-test-images.yaml
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-03-09 13:55:35 +08:00
ThreadDao
7a3223dc81
tags collection stats
...
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2021-03-08 10:09:48 +08:00
sunby
ddddd65d10
Delete internal/errors package
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-03-05 10:15:27 +08:00
godchen
3ba0dc2f44
Change doc
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-03-04 10:35:28 +08:00
zhenshan.cao
966cf42b54
Enable indexnode to build index asynchronously
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-27 10:45:03 +08:00
godchen
7b9fdd7f29
Add opentracing
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-02-26 17:44:24 +08:00
zhenshan.cao
6b392cbe58
Fix memory leak
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-23 14:35:00 +08:00
sunby
ea1d9ea99a
Replace log in dataservice
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-23 09:58:06 +08:00
dragondriver
68518fec38
Fix memory leak in indexnode
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-02-22 18:33:40 +08:00
dragondriver
da94fbed0a
Parse to map when the key of index params is named params
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-02-07 21:32:37 +08:00
bigsheeper
01e9dc8e3f
Remove collection name
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 11:52:19 +08:00
bigsheeper
5e781b9370
Remove field name in query node and segCore
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 10:10:07 +08:00
sunby
45b99c0cf3
Save indexName and indexID in IndexCodec
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-02 19:56:11 +08:00
zhenshan.cao
bfa14d40a8
Rename indexID to indexBuildID and add IndexID and IndexName
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-02 19:56:04 +08:00
neza2017
3a866dab3d
Update proto
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-28 17:25:43 +08:00
neza2017
2ebeade25e
Fix get component states
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-28 17:13:00 +08:00
sunby
2be8cc1c4b
Add index params serde to IndexCodec
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-28 16:41:24 +08:00
cai.zhang
f940cc455a
Add dockerfile for index
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-26 09:38:40 +08:00
zhenshan.cao
aef4e41e91
Enable proxynode microbatch pulsar msg and enable parallel test
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-23 20:58:46 +08:00
xige-16
d22c2cd203
Add proto for query service loading collection
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-18 19:32:08 +08:00
quicksilver
7fe061b846
Raneme init_devcontainer.sh to devcontainer.sh
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-15 14:38:36 +08:00
neza2017
56ab2bda2d
Update proto:add partition id
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-15 11:53:22 +08:00
zhenshan.cao
63b21321d6
Refactor indexservice and update doc
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-15 13:50:27 +08:00