Commit Graph

121 Commits

Author SHA1 Message Date
dragondriver
fb98f06de8
Expose simd type to index node metrics (#8300)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-23 11:29:54 +08:00
Xiaofan
f5173b595f
Add a channel prefix for all channels (#8166)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-23 10:53:53 +08:00
dragondriver
78fe530d87
Initialize the simd type when initialize an indexnode (#8276)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-22 16:05:59 +08:00
Xiaofan
0c35cd57b0
Unify param table to the same naming format (#8254)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-20 23:27:52 +08:00
Xiangyu Wang
c314b2b4a8
Remove unused label rules (#7929)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-18 18:19:51 +08:00
cai.zhang
a6b6d16381
Delete unused code in indexnode (#8169)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-18 11:05:50 +08:00
cai.zhang
8d5dc5ba07
Add comment for CreateIndex in indexnode (#8168)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-18 00:01:51 +08:00
cai.zhang
fa00f1f48a
It is also correct to get the failed index status in indexcoord ut (#8059)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-16 19:56:41 +08:00
cai.zhang
4814f9cb1c
Add opentrace log for index component (#7946)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-16 10:07:48 +08:00
cai.zhang
b965767e5c
Increase code coverage rate for indexcoord (#7857)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-14 14:09:48 +08:00
cai.zhang
63a83f027f
Reduce running unit test time for indexcoord (#7779)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-14 10:41:21 +08:00
cai.zhang
b1805d980c
Reduce running unit test time for indexnode (#7773)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-13 12:00:08 +08:00
cai.zhang
be9427e8b9
Increase covd coverage for paramtable (#7583)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-10 14:14:12 +08:00
cai.zhang
f6621dc847
Increase code coverage for indexnode component (#7520)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Co-authored-by: edward.zeng <jie.zeng@zilliz.com>
2021-09-09 17:09:08 +08:00
dragondriver
9b21b3b5fd
Initialize test framework of proxy (#7563)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-09 10:06:29 +08:00
cai.zhang
e1081b6783
Increase indexcoord component code coverage (#7558)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-08 14:23:59 +08:00
cai.zhang
e6e03fc93f
Add unittest for index component (#7475)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-06 17:54:41 +08:00
dragondriver
960741ee16
Support more detailed metrics information (#7472)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-03 19:35:47 +08:00
dragondriver
648d22ee40
Expose metrics of IndexCoord and IndexNodes (#7157)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-19 10:28:10 +08:00
cai.zhang
382fa6f274
IndexNode dose not need to wait for IndexCoord to start to comlete (#7074)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-08-13 18:12:08 +08:00
XuanYang-cn
6c5be83ea1
Refactor NewEtcdKV API (#7075)
This PR mainly did the following 3 things:

- Refactor NewEtcdKV API
- Add unittests for `kv/etcd/`, raise the coverage
up to 94%
- Correct some APIs from version to revision

The old NewEtcdKV() API took `clientv3.Client` as
one of the input, making everyone who using this API
had to create a new `clientv3.Client`. So there're a lot
of repeated codes everywhere.

Meanwhile, someplace didn't even check whether the
`clientv3.New()` returned error or not, and the old API
also didn't check whether the clientv3.Client was Nil.
This caused issue#6955.

Ectd's version and revision have nothing alike, they
represent different concepts, but our APIs kind of
mixed them all up. So I correct those APIs' names.

Resolves: #6955

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-13 11:04:09 +08:00
Xiangyu Wang
5e78fd6a97
[skip ci]Update approvers (#7063)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-08-12 19:36:09 +08:00
cai.zhang
d51fec6678
Enhance log when create index failed (#6871)
* Enhance log when create index failed

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for repeatedly drop index

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-29 14:47:22 +08:00
godchen
db94d7771f
Read vector from disk (#6707)
* Read vector from disk

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix git action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix test error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix caculate error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change var name

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix len error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused code

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change bytes to float method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change float to bytes method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-24 09:25:22 +08:00
dragondriver
2821c08560
Add unittests for metric_type: tanimoto (#6773)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-23 20:59:33 +08:00
cai.zhang
e1b52b0583
Add unittest for IndexCoord and IndexNode (#6698)
* Add unittest for IndexCoord and IndexNode

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix verifiers

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-23 10:44:12 +08:00
cai.zhang
9aa56cd7b8
Add log for IndexCoord and IndexNode (#6597)
* Add log for IndexCoord and IndexNode

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Make verifier

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Delete unused script

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-22 11:40:11 +08:00
cai.zhang
f469a315d6
Refactor the logic of assign tasks in IndexCoord (#6328)
* Refactor the logic of assign tasks in IndexCoord

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bugs

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for unittest

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add lock for map

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Improve code

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix unittest bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reduce duriation for timetick

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Update orm version

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reset sdk version

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reset orm version

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reset test ip

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for unissued

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Rename some variables

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Use break instead of continue in select::case

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-14 14:15:55 +08:00
Cai Yudong
a7a9eeeffe
Add time record for build index and search process (#6231)
* add timerecord for build index process

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix build error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add timerecord for query process

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-30 19:46:14 +08:00
neza2017
bf068d84ad
fix index coord (#6042)
* fix index coord

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* remove memstream from RmsFactory

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-24 14:36:08 +08:00
godchen
99be4c09a5
change retry (#5996)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-23 09:24:10 +08:00
Cai Yudong
7dbf33b788
Update component names in config files (#5989)
* rename master/indexService in .yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxyNode to proxy in config files

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move master.yaml to root_coord.yaml, move proxy_node.yaml to proxy.yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proto/query_service.proto to proto/query_coord.proto

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proxy_node_manager.go to proxy_manager.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add query_coord.pb.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-22 19:08:03 +08:00
Cai Yudong
92e429d812
Rename IndexService to IndexCoord (#5932)
* rename package indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix queryservice static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move distributed/indexservice to distributed/indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move internal/indexservice to internal/indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename MasterComponent to RootCoordComponent

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for queryservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for dataservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for datanode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for proxynode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for querynode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename IndexService to IndexCoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename IndexService to IndexCoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix rebase issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-21 17:28:03 +08:00
XuanYang-cn
ab7f642740
Add alias in paramtable (#5878)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-19 12:38:06 +08:00
zhenshan.cao
5c69c6902d Refactor proxy channelTimeTick and fix search timeout (#5760)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:36:36 +08:00
Cai Yudong
0f4bd50ce3 Support ETCD endpoints array (#5755) 2021-06-15 16:36:36 +08:00
zhenshan.cao
33a43f48c8 [skip ci] Fix bug: Search timeout and refactor client init logic (#5735)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:24:48 +08:00
zhenshan.cao
bc46e4780d
Fixbug:index is not loaded in some cases (#5633)
* Fixbug:index is not created in some cases

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Delete unecessary print

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* fix create index

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-06 09:41:35 +08:00
zhenshan.cao
03e3b4beb4
Add log to startup process (#5594)
* Add log to startup process

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fixbug: avoid ctx exceed deadline

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fmt code

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fix bug: wrong time unit

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fixbug: datanservice client reconnect use time ctx

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-04 16:29:35 +08:00
FluorineDog
9a90313390
Support GetEntityByIDs in CGo, fix segcore bugs (#5563)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-04 10:38:34 +08:00
Xiangyu Wang
f791ccb246 Update OWNERS file
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-06-03 20:01:24 +08:00
zhenshan.cao
b0b8f58192
Fix bug: search timeout (#5557)
* Fix bug: search timeout

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Add log and fix unittest bug

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-03 14:58:34 +08:00
zhenshan.cao
4af2cacb28
Refactor IndexService and IndexNode: remove redundant serviceID (#5491)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-05-31 10:32:30 +08:00
Xiangyu Wang
ca0993c274 Update OWNERS
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-30 18:58:47 +08:00
zhenshan.cao
67c4c915b7
Move debugging code of index query to a separate module (#5490)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-05-29 17:07:29 +08:00
cai.zhang
a606ab9c67
Add power-off restart logic for IndexService and IndexNode (#5395)
Add power-off restart logic for IndexService and IndexNode


Signed-off-by: <xiaocai2333> <cai.zhang@zilliz.com>
2021-05-27 14:24:29 +00:00
neza2017
f80dbab6ec
watch proxynode (#5424)
watch proxy node and,
send InvalidataCollectionMetaCache to each proxynode

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Resolves: #5423
2021-05-26 12:14:30 +00:00
neza2017
c28c34e852
let master use session.ServerID as nodeID (#5385)
let master use session.ServerID as nodeID

also see #5386 

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-25 07:06:05 +00:00
neza2017
e326eaad1e
master client,reconnect if grpc failed (#5360)
master client,reconnect if grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-24 01:44:49 +00:00
godchen
0eee7e5252
Change session (#5329)
Change session.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-21 11:28:52 +00:00