Commit Graph

180 Commits

Author SHA1 Message Date
congqixia
7a2546c24a
Fix proxy panic if quit before init (#5981)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-22 17:40:07 +08:00
xige-16
2af1c27811
Rename queryservice to querycoordinator (#5931)
* Rename queryservice to querycoordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix param table

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix unittest

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* service to coordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* set querycoord in ci

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-22 16:44:09 +08:00
Cai Yudong
2926a78968
Rename proxynode to proxy (#5961)
* rename package name from proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move distributed/proxynode to distributed/proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move internal/proxynode to internal/proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-22 14:40:07 +08:00
quicksilver
7fe061b846 Raneme init_devcontainer.sh to devcontainer.sh
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-15 14:38:36 +08:00
neza2017
56ab2bda2d Update proto:add partition id
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-15 11:53:22 +08:00
zhenshan.cao
63b21321d6 Refactor indexservice and update doc
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-15 13:50:27 +08:00
GuoRentong
f3d2afa4f4 Update doc:detailed interface definition
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-11 18:35:54 +08:00
quicksilver
a6690dbc99 Fix syntax error: Bad fd number
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-11 15:25:19 +08:00
yukun
18cfd31d40 Fix Load and Remove with prefix
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-11 15:17:06 +08:00
sunby
5dfe9448ae Modify test_index
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-08 16:00:16 +08:00
GuoRentong
82ad488555 Update doc: rocksmq
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-08 18:37:24 +08:00
XuanYang-cn
65089ea362 Fix flush sync test bug and remove unecessary lock in meta_table
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-08 17:20:59 +08:00
GuoRentong
c4599e26de Update doc: node api
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-08 16:59:19 +08:00
bigsheeper
e13822043d Fix search test when partition not exists, add binary vector search validation
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-08 16:41:27 +08:00
godchen
fa73f17f94 Add opentracing doc
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-08 15:32:38 +08:00
dragondriver
ced56c84f8 Add support to build binary vector index
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-07 14:56:17 +08:00
godchen
8f1f118b26 Add query opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-07 14:24:35 +08:00
bigsheeper
a1e2ff3c26 Add gcNode to write node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-06 16:44:12 +08:00
neza2017
4015d7245d Merge operation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-06 14:45:50 +08:00
zhenshan.cao
b06e01f523 Initmate querynode collection manager
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-06 11:31:36 +08:00
godchen
1c87faf8b3 Add insert request opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-06 11:17:35 +08:00
XuanYang-cn
c0797333ce Add ddl binlog metaTable in writenode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41 Add test for master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 16:19:42 +08:00
zhenshan.cao
a0244b7683 Add segment expire logic for proxy
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-24 16:53:31 +08:00
XuanYang-cn
32977e270c Add impl cgo of parquet
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-08 14:41:04 +08:00
zhenshan.cao
7bbbc14637 Fix bug: address already used
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-07 15:22:20 +08:00
xige-16
37d7526d31 Fix search test failure
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-07 14:37:42 +08:00
neza2017
aeca8f85a5 Run cwrapper unittest in ci
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 17:39:58 +08:00
cai.zhang
806a97505c Add query node for CI progress
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-05 16:12:45 +08:00
neza2017
70710dee47 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 16:11:03 +08:00
cai.zhang
18a9ffb617 Increase timeout for golangci-lint
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-04 17:57:24 +08:00
zhenshan.cao
0b6fe0a8cf Refactor paramtable
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-04 14:18:17 +08:00
godchen
7ab5b5d80d Add MinIO kv implements
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-03 19:00:11 +08:00
neza2017
cec903da19 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-02 18:31:56 +08:00
zhenshan.cao
23aebe7e2a Fix proxy unittest
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-01 19:53:53 +08:00
neza2017
17cc7bc337 Use MetaRootPath instead
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-01 17:03:39 +08:00
dragondriver
84b15c1f5c Fix the bug of meta cache in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-01 16:41:25 +08:00
cai.zhang
c1fbc4f8fa Open the tests for insert
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-01 15:18:57 +08:00
FluorineDog
6b74d82259 Support some validation for dsl parser
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-01 02:16:53 +08:00
dragondriver
72a7af3f4a Add time tick logic to Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-26 17:47:46 +08:00
neza2017
32445dd02b Add cpu profile
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 16:01:31 +08:00
cai.zhang
57da66ba72 Update read config logic in docker
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-26 15:18:36 +08:00
zhenshan.cao
045fc3d808 Fix segment fault
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-26 14:29:01 +08:00
XuanYang-cn
0a16a9a651 Add logic of getting milvus distributed config from master by grpc
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-26 13:21:06 +08:00
zhenshan.cao
f12366342f Fix bug and delete unused code
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-25 16:45:42 +08:00
bigsheeper
3b24c52a8c Catch cpp runtime error
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-25 16:20:07 +08:00
dragondriver
a7b3efecd7 Read query node num from param table instead of hardcode
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-25 16:17:33 +08:00
bigsheeper
41fb2d2991 fix lock usage
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-24 16:12:39 +08:00
bigsheeper
8ec7e4576e Add dropPartition and createPartition
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-24 15:59:51 +08:00
neza2017
ad0078c07a Load grpc port of proxy from config
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-24 11:50:21 +08:00
dragondriver
c7a49c97cc Add repack function of InsertTask in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-23 16:52:17 +08:00
zhenshan.cao
7c39fc24a9 Simplify proxy config
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-21 19:12:59 +08:00
quicksilver
859ff62bc9 Unify Network for UnitTest
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-11-21 16:54:20 +08:00
cai.zhang
8ff82c2fd5 Add judgement when executing clang-format
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-21 16:37:48 +08:00
dragondriver
e36b565ec0 Normalize the naming of variable and function in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-21 16:31:38 +08:00
cai.zhang
17c3a70d64 update ParameterTable
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-21 15:06:46 +08:00
dragondriver
aabe9afd75 Add meta cache in Proxy and use meta cache to generate id in InsertTask
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-20 17:53:31 +08:00
zhenshan.cao
50d59d91ee Add segmentIDAssigner for proxy
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-19 21:02:31 +08:00
dragondriver
0b2add0d7b Fix query result loop logic in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-19 18:45:24 +08:00
neza2017
c3de667bf5 Implement segment management in master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-18 19:46:18 +08:00
bigsheeper
99ea7e9f05 Update param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-18 19:33:56 +08:00
sunby
9e4ce8454e Implement segment management in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-18 15:04:17 +08:00
dragondriver
a18ac17a98 Increase timeout interval of task in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-18 15:44:17 +08:00
xige-16
4e8efe85df Delete wrong useage of *TsMsg
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-11-17 14:10:07 +08:00
cai.zhang
5bcabffdaf Replace conf struct with ParamsTable
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-16 21:10:43 +08:00
dragondriver
aff1e79f82 Fix SearchTask in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-16 17:01:10 +08:00
dragondriver
23d9ddb4a3 Add unittest to grpc service that Proxy provide
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-14 18:18:10 +08:00
dragondriver
c2382adf04 Initialize the task scheduler of Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-14 11:24:49 +08:00
cai.zhang
0b478b614a Change Id to ID in go files according to golint check
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-13 15:17:18 +08:00
XuanYang-cn
88cb274644 Refactor master scheduler: drop_collection
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-13 12:46:24 +08:00
cai.zhang
1cd2fd9954 Add the standard of golangci-lint
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-12 12:04:12 +08:00
bigsheeper
c9c8fb9def Close pulsar channel
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-12 11:18:23 +08:00
cai.zhang
3b2c86c42d Fix bug for cmake_build dir was changed
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-12 10:45:46 +08:00
zhenshan.cao
057563d936 Refactor master
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-11 17:53:54 +08:00
XuanYang-cn
a2289d3ce6 Add softTimeTickBarrier and hardTimeTickBarrier
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-11 16:41:53 +08:00
dragondriver
fb1e24ade8 Refactor task interface of Proxy and fix the wait logic of task
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-11 16:00:56 +08:00
zhenshan.cao
3b5765ae47 Add proxy timetick
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-11 09:54:01 +08:00
zhenshan.cao
87a4495c59 Fix proxy bug
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-09 17:52:47 +08:00
dragondriver
b8ccbc1c97 Add InsertTask and SearchTask to Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-07 16:18:23 +08:00
dragondriver
c732c267e1 Refactor proxy framework and add CreateCollection Task
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-05 18:01:33 +08:00