Commit Graph

284 Commits

Author SHA1 Message Date
zhenshan.cao
c2734fa55f Fix bug and enchance system
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-03-22 16:36:10 +08:00
dragondriver
af32f442bb Split big insert message into serveral smaller
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-03-09 16:18:58 +08:00
sunby
aeb3038545 Remove GetCount in data service
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-03-08 10:49:44 +08:00
sunby
ddddd65d10 Delete internal/errors package
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-03-05 10:15:27 +08:00
godchen
7b9fdd7f29 Add opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-02-26 17:44:24 +08:00
sunby
ea1d9ea99a Replace log in dataservice
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-23 09:58:06 +08:00
sunby
b07a371f25 Create a goroutine to receive msg for every consumer
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-22 09:58:34 +08:00
neza2017
1d3fdebcf4 Add drop index for proxy
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-20 15:38:44 +08:00
sunby
5769c000a1 Find failed python tests and add skip mark "r0.3-test"
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-19 15:37:04 +08:00
bigsheeper
d8f41db823 Fix search crash error
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-19 13:56:01 +08:00
sunby
5b61182140 Add zap log
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-18 19:53:55 +08:00
groot
dc3736281a Integrate message stream
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-02-08 14:30:54 +08:00
Xiangyu Wang
fd9e42d3ed Update msgstream/unmarshal interface
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-02-03 17:30:10 +08:00
bigsheeper
abffdbff62 Remove partition tag
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 16:44:38 +08:00
yukun
35f6f4b7c2 Add consume in rmq_msgstream
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-02-03 15:18:54 +08:00
Xiangyu Wang
99cef4b5c8 Add Distributed main entrypoint
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-01-29 17:08:31 +08:00
dragondriver
eba6bb288c Refactor the sructure of proto file
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-22 09:36:18 +08:00
xige-16
d22c2cd203 Add proto for query service loading collection
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-18 19:32:08 +08:00
XuanYang-cn
37688559f1 Fix multi rowdata writenode crash bug
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-16 15:06:19 +08:00
GuoRentong
f3d2afa4f4 Update doc:detailed interface definition
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-11 18:35:54 +08:00
quicksilver
a6690dbc99 Fix syntax error: Bad fd number
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-11 15:25:19 +08:00
yukun
18cfd31d40 Fix Load and Remove with prefix
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-11 15:17:06 +08:00
sunby
5dfe9448ae Modify test_index
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-08 16:00:16 +08:00
yukun
d11f00cd9f Add rocksdb in kv
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-08 19:07:57 +08:00
GuoRentong
82ad488555 Update doc: rocksmq
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-08 18:37:24 +08:00
XuanYang-cn
65089ea362 Fix flush sync test bug and remove unecessary lock in meta_table
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-08 17:20:59 +08:00
GuoRentong
c4599e26de Update doc: node api
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-08 16:59:19 +08:00
bigsheeper
e13822043d Fix search test when partition not exists, add binary vector search validation
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-08 16:41:27 +08:00
godchen
fa73f17f94 Add opentracing doc
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-08 15:32:38 +08:00
dragondriver
ced56c84f8 Add support to build binary vector index
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-07 14:56:17 +08:00
godchen
8f1f118b26 Add query opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-07 14:24:35 +08:00
bigsheeper
a1e2ff3c26 Add gcNode to write node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-06 16:44:12 +08:00
neza2017
4015d7245d Merge operation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-06 14:45:50 +08:00
XuanYang-cn
d64aa37db3 Add KFlush msg type
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-11 12:01:20 +08:00
neza2017
32445dd02b Add cpu profile
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 16:01:31 +08:00
cai.zhang
57da66ba72 Update read config logic in docker
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-26 15:18:36 +08:00
zhenshan.cao
045fc3d808 Fix segment fault
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-26 14:29:01 +08:00
XuanYang-cn
0a16a9a651 Add logic of getting milvus distributed config from master by grpc
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-26 13:21:06 +08:00
zhenshan.cao
f12366342f Fix bug and delete unused code
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-25 16:45:42 +08:00
bigsheeper
41fb2d2991 fix lock usage
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-24 16:12:39 +08:00
bigsheeper
8ec7e4576e Add dropPartition and createPartition
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-24 15:59:51 +08:00
neza2017
ad0078c07a Load grpc port of proxy from config
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-24 11:50:21 +08:00
zhenshan.cao
50d59d91ee Add segmentIDAssigner for proxy
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-19 21:02:31 +08:00
neza2017
429c94e18b Add grpc for proxy
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-19 17:09:22 +08:00
neza2017
ce89f64bb3 Set master config
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-19 10:46:17 +08:00
dragondriver
c2382adf04 Initialize the task scheduler of Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-14 11:24:49 +08:00
bigsheeper
c9c8fb9def Close pulsar channel
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-12 11:18:23 +08:00
cai.zhang
3b2c86c42d Fix bug for cmake_build dir was changed
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-12 10:45:46 +08:00
bigsheeper
b4bd9cf9d6 Add collection, segment, and partition unittest
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-11 11:22:19 +08:00
zhenshan.cao
86d2d36053 Add MemoryKV, mockkv and tso test
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-06 16:47:18 +08:00
zhenshan.cao
9c490a209b Delete unused code
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-05 19:11:12 +08:00
neza2017
10217f9485 Add test for etcd kv
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-02 10:26:33 +08:00
zhenshan.cao
251bc2a19e Fix conf.LoadConfig failed to load yaml file
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-19 18:31:00 +08:00
GuoRentong
974496a2b2 Update issue templates
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2020-10-19 17:45:56 +08:00
zhenshan.cao
6a136c1a76 Change import path
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-10-19 17:39:51 +08:00
shengjh
3d7181617b Add ci and format code for proxy
Signed-off-by: shengjh <1572099106@qq.com>
2020-10-15 16:32:22 +08:00
xige-16
d7e6b99394 Add config reader
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-10-13 15:03:46 +08:00
neza2017
bb517eeca1 Add test for proxy-go
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-10-12 13:54:00 +08:00
bigsheeper
d4e485d37a Add pulsar authentication
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-10-10 11:06:07 +08:00
neza2017
b0189c2450 Update time sync
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-10-09 15:37:24 +08:00
zhenshan.cao
32157159a2 Fix bug:only need to add unittest when build with -u option
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-24 11:18:31 +08:00
cai.zhang
046b74e8b8 Add the interface of S3 for storage
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-09-24 10:58:20 +08:00
shengjh
70f5a44f8f Use async send in pulsar producer
Signed-off-by: shengjh <1572099106@qq.com>
2020-09-21 19:28:42 +08:00
rain
a95f6843aa Add create index interface and update the proto of master
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-21 15:10:54 +08:00
FluorineDog
c123b57e98 Support Benchmark
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-09-21 10:32:20 +08:00
rain
6bc7e6d372 Update collection and segment id type and Refactor channel controller
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-18 15:55:33 +08:00
zhenshan.cao
3acd55f675 Fix config path error
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-18 01:53:18 +08:00
FluorineDog
a177f48577 Support Query Dispatch, enable offset for bitset
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-09-17 18:37:54 +08:00
rain
7f798c5fce Synchronous creation of collection instead of goroutine
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-17 10:00:03 +08:00
bigsheeper
2474d06a10 Add index building of segment
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-09-16 18:19:29 +08:00
zhenshan.cao
be58ccb65e Delete unused
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-12 16:57:37 +08:00
shengjh
4cf6e079bc Add etcd watcher
Signed-off-by: shengjh <1572099106@qq.com>
2020-09-11 18:55:43 +08:00
xige-16
f319803eab Fix hardcode load config path
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-09-09 19:09:02 +08:00
zhenshan.cao
2863029665 Fix config
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-09 18:59:56 +08:00
bigsheeper
1370da9498 Add message sending and key2segment, use hard code schema
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-09-09 15:24:07 +08:00
xige-16
81e70ee6c9 Hash message to partitioned topic
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-09-07 21:07:36 +08:00
zhenshan.cao
a90b725ba8 Add timestamp for key2segment message
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-09-07 19:46:14 +08:00
rain
642554e7bd Refactor directory structure and update the default value of server
Signed-off-by: rain <boyan.wang@zilliz.com>
2020-09-07 19:19:50 +08:00
xige-16
d58f6ff817 Support write node send key2seg msg
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-09-06 19:39:35 +08:00
become-nice
558a81dcbe Update the interface of writenode
Signed-off-by: become-nice <995581097@qq.com>
2020-09-01 18:11:43 +08:00
bigsheeper
bb9c906ef6 Use go mod instead of GO_PATH and add more cgo interfeces
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-09-01 16:23:39 +08:00
cai.zhang
26805ebd3a Add interface for minio and add grpc proto for master
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-08-29 17:40:29 +08:00
bigsheeper
6d777b7518 Init reader component
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-08-25 15:38:35 +08:00
become-nice
b8f012d8f0 Add interface of writenode
Signed-off-by: become-nice <995581097@qq.com>
2020-08-24 11:58:51 +08:00