Commit Graph

251 Commits

Author SHA1 Message Date
sunby
7c0c835b49 Init params and add segment start/end position
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-25 15:16:28 +08:00
XuanYang-cn
5015a1035d Add logic of get meta from MasterService for datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-25 17:10:32 +08:00
neza2017
1fe099775d Refactor data service
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-25 15:17:17 +08:00
neza2017
e89e89b894 Add main for masterservice
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-25 12:03:44 +08:00
dragondriver
cd52adc18b Add proxy service to ci workflow
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-22 12:57:23 +08:00
dragondriver
eba6bb288c Refactor the sructure of proto file
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-22 09:36:18 +08:00
cai.zhang
83c60fd84f Update proto about index
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-21 15:20:23 +08:00
cai.zhang
7619d968fd Generate random node port for indexService
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-20 18:26:20 +08:00
yukun
527c0c49df Add rocksmq unittest with goroutines
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-20 16:46:58 +08:00
cai.zhang
c35079d7e7 Update registerNode in indexservice
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-20 10:15:43 +08:00
bigsheeper
f02bd8c8f5 Rename query node package, implement watchDmChannel
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-16 10:12:14 +08:00
bigsheeper
b98b226d8c Refactor query node and implement channelRemove, channelAdd
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-15 15:28:54 +08:00
quicksilver
7fe061b846 Raneme init_devcontainer.sh to devcontainer.sh
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-15 14:38:36 +08:00
neza2017
56ab2bda2d Update proto:add partition id
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-15 11:53:22 +08:00
zhenshan.cao
63b21321d6 Refactor indexservice and update doc
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-15 13:50:27 +08:00
yukun
51ca741a49 Add singlenode launch main
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-14 09:52:03 +08:00
neza2017
4015d7245d Merge operation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-06 14:45:50 +08:00
cai.zhang
1cfc2ff0a5 Add make clean in Makefile
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-06 13:50:48 +08:00
zhenshan.cao
b06e01f523 Initmate querynode collection manager
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-06 11:31:36 +08:00
godchen
1c87faf8b3 Add insert request opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-06 11:17:35 +08:00
FluorineDog
e16a7478da Add verifier
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-05 20:13:54 +08:00
FluorineDog
eb8717188e Enable load index
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-28 17:55:48 +08:00
FluorineDog
e65cfe1e3d Enable complex dsl parser
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-19 10:36:49 +08:00
zhenshan.cao
0d75840ed6 Fix missing file
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-18 18:35:03 +08:00
neza2017
0110ba6bd2 List events of binlog
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-12 16:52:09 +08:00
zhenshan.cao
fc054874d3 Add index_builder
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-10 17:55:55 +08:00
bigsheeper
762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-10 16:31:09 +08:00
XuanYang-cn
32977e270c Add impl cgo of parquet
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-08 14:41:04 +08:00
zhenshan.cao
7bbbc14637 Fix bug: address already used
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-07 15:22:20 +08:00
xige-16
37d7526d31 Fix search test failure
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-07 14:37:42 +08:00
neza2017
aeca8f85a5 Run cwrapper unittest in ci
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 17:39:58 +08:00
cai.zhang
806a97505c Add query node for CI progress
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-05 16:12:45 +08:00
neza2017
70710dee47 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 16:11:03 +08:00
zhenshan.cao
0b6fe0a8cf Refactor paramtable
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-04 14:18:17 +08:00
neza2017
32445dd02b Add cpu profile
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 16:01:31 +08:00
cai.zhang
57da66ba72 Update read config logic in docker
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-26 15:18:36 +08:00
zhenshan.cao
045fc3d808 Fix segment fault
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-26 14:29:01 +08:00
XuanYang-cn
0a16a9a651 Add logic of getting milvus distributed config from master by grpc
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-26 13:21:06 +08:00
sunby
f4c643f1bd Add number limitation of partition and field. And Change default partition tag to "_default"
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-25 18:39:05 +08:00
bigsheeper
84f3d974fa Delete pulsar address test and refactor master param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-23 18:04:09 +08:00
FluorineDog
76001b29b6 Support Range Query
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-23 11:56:45 +08:00
sunby
578e952c60 Add segment threshold factor
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-23 10:20:30 +08:00
quicksilver
859ff62bc9 Unify Network for UnitTest
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-11-21 16:54:20 +08:00
cai.zhang
8ff82c2fd5 Add judgement when executing clang-format
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-21 16:37:48 +08:00
dragondriver
e36b565ec0 Normalize the naming of variable and function in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-21 16:31:38 +08:00
cai.zhang
17c3a70d64 update ParameterTable
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-21 15:06:46 +08:00
bigsheeper
dd3654f953 Fix query node param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-20 17:10:24 +08:00
quicksilver
1728a4f419 Update .devcontainer.json
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-11-20 15:02:26 +08:00
neza2017
cda47a9252 Remove unused code
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-20 09:13:29 +08:00
neza2017
ce89f64bb3 Set master config
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-19 10:46:17 +08:00
neza2017
c3de667bf5 Implement segment management in master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-18 19:46:18 +08:00
bigsheeper
99ea7e9f05 Update param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-18 19:33:56 +08:00
sunby
9e4ce8454e Implement segment management in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-18 15:04:17 +08:00
neza2017
544c701f98 Start timeSyncMsgProducer in master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-18 10:14:11 +08:00
dragondriver
b32e55d5f0 Add timeout logic to task in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-17 20:00:23 +08:00
quicksilver
72ed65d7f8 Check builder images
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-11-17 18:36:03 +08:00
bigsheeper
e117a6097a Fix pulsar url
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-17 16:32:34 +08:00
cai.zhang
5bcabffdaf Replace conf struct with ParamsTable
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-16 21:10:43 +08:00
dragondriver
aff1e79f82 Fix SearchTask in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-16 17:01:10 +08:00
FluorineDog
8c48cf30c0 Format and and extra parameters for checking
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-16 15:41:56 +08:00
neza2017
9d3a21a9c9 Rewrite NewTSOKVBase
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-16 12:37:46 +08:00
zhenshan.cao
9f948d3095 Remove mockmaster
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-14 16:40:10 +08:00
dragondriver
c2382adf04 Initialize the task scheduler of Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-14 11:24:49 +08:00
cai.zhang
1cd2fd9954 Add the standard of golangci-lint
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-12 12:04:12 +08:00
bigsheeper
c9c8fb9def Close pulsar channel
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-12 11:18:23 +08:00
cai.zhang
3b2c86c42d Fix bug for cmake_build dir was changed
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-12 10:45:46 +08:00
neza2017
d71083bf6b Change master scheduler: CreateCollection
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-11 20:01:04 +08:00
zhenshan.cao
3b5765ae47 Add proxy timetick
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-11 09:54:01 +08:00
zhenshan.cao
87a4495c59 Fix proxy bug
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-09 17:52:47 +08:00
zhenshan.cao
db9957d48b Add unittest for master
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-07 18:03:52 +08:00
dragondriver
b8ccbc1c97 Add InsertTask and SearchTask to Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-07 16:18:23 +08:00
zhenshan.cao
9c490a209b Delete unused code
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-05 19:11:12 +08:00
dragondriver
c732c267e1 Refactor proxy framework and add CreateCollection Task
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-05 18:01:33 +08:00
zhenshan.cao
e9122921b9 Use type alias
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-04 17:58:43 +08:00
neza2017
d18c17e786 Add Meta table implementation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-04 16:28:14 +08:00
godchen
bade037fb2 Add meta_table implementation
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-11-04 16:01:28 +08:00
zhenshan.cao
da18bc4952 Update proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-03 16:38:16 +08:00
zhenshan.cao
e962b57fbb Add timestamp allocator
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-03 14:53:36 +08:00
neza2017
10217f9485 Add test for etcd kv
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-02 10:26:33 +08:00
zhenshan.cao
f0cc9c8ad3 Add tso for master and add timestamp
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-29 09:31:08 +08:00
quicksilver
eb64839aef Update build environment
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-10-26 15:45:18 +08:00
zhenshan.cao
251bc2a19e Fix conf.LoadConfig failed to load yaml file
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-19 18:31:00 +08:00
GuoRentong
974496a2b2 Update issue templates
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2020-10-19 17:45:56 +08:00
zhenshan.cao
6a136c1a76 Change import path
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-19 17:39:51 +08:00
zhenshan.cao
64295db471 Refact master and proxy and add etcdutil
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-15 21:31:50 +08:00
shengjh
3d7181617b Add ci and format code for proxy
Signed-off-by: shengjh <1572099106@qq.com>
2020-10-15 16:32:22 +08:00
xige-16
d7e6b99394 Add config reader
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-10-13 15:03:46 +08:00
cai.zhang
046b74e8b8 Add the interface of S3 for storage
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-09-24 10:58:20 +08:00
rain
a95f6843aa Add create index interface and update the proto of master
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-21 15:10:54 +08:00
FluorineDog
c123b57e98 Support Benchmark
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-09-21 10:32:20 +08:00
shengjh
5aee3352ce Add storage benchmark
Signed-off-by: shengjh <1572099106@qq.com>
2020-09-19 18:16:30 +08:00
zhenshan.cao
3acd55f675 Fix config path error
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-18 01:53:18 +08:00
FluorineDog
a177f48577 Support Query Dispatch, enable offset for bitset
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-09-17 18:37:54 +08:00
zhenshan.cao
3bb0da1f57 Add config logic and tikv_benchmark.go
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-17 14:31:50 +08:00
bigsheeper
1370da9498 Add message sending and key2segment, use hard code schema
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-09-09 15:24:07 +08:00
shengjh
8f9a8d83e4 Enable timesyc
Signed-off-by: shengjh <1572099106@qq.com>
2020-09-09 14:34:03 +08:00
rain
e704667bc5 Finish all feature and update the README of master
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-08 16:57:48 +08:00
xige-16
81e70ee6c9 Hash message to partitioned topic
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-09-07 21:07:36 +08:00
zhenshan.cao
a90b725ba8 Add timestamp for key2segment message
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-07 19:46:14 +08:00
rain
642554e7bd Refactor directory structure and update the default value of server
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-07 19:19:50 +08:00
become-nice
b8f012d8f0 Add interface of writenode
Signed-off-by: become-nice <995581097@qq.com>
2020-08-24 11:58:51 +08:00