Commit Graph

221 Commits

Author SHA1 Message Date
zhenshan.cao
c2734fa55f Fix bug and enchance system
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-03-22 16:36:10 +08:00
Xiangyu Wang
9ba9bf5e8a Release memory after rocksdb.write()
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-03-19 20:20:32 +08:00
yukun
0823382876 Refactor rmq_msgstream
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-03-19 19:33:21 +08:00
yukun
207f6ae773 Fix rocksmq CreateConsumerGroup
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-03-16 21:37:25 +08:00
XuanYang-cn
32660d547b Add endPositions in datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-03-16 17:55:42 +08:00
Xiangyu Wang
fc6472eb08 Implement RocksMQ client
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-03-13 11:46:50 +08:00
godchen
f3649f0419 Refactor interface and proto
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-03-12 14:22:09 +08:00
xige-16
09ae985daa Fix wrong error code in master_service_test.go
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-10 22:06:22 +08:00
del-zhenwu
b1763c3690 Disable xdist in test cases
Signed-off-by: del-zhenwu <zhenxiang.li@zilliz.com>
2021-03-10 15:27:26 +08:00
neza2017
7b2d67242d Fix describe segment if index not exist
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-10 14:45:35 +08:00
Xiangyu Wang
c875fdf913 Add logs in rocksmq
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-03-10 10:02:55 +08:00
yukun
0d499a9f99 Fix search waiting for seconds in singlenode
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-03-09 17:42:13 +08:00
quicksilver
455cc59dbb Update publish-test-images.yaml
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-03-09 13:55:35 +08:00
ThreadDao
7a3223dc81 tags collection stats
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2021-03-08 10:09:48 +08:00
quicksilver
74154a11a4 Fix deploy error during regression stage
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-03-05 16:52:45 +08:00
sunby
ddddd65d10 Delete internal/errors package
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-03-05 10:15:27 +08:00
godchen
3ba0dc2f44 Change doc
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-03-04 10:35:28 +08:00
bigsheeper
456d0e8c23 Refactor time tick align strategy
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-27 18:33:29 +08:00
BossZou
7893ebdb40 Fix test insert case
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
2021-02-26 17:59:13 +08:00
godchen
7b9fdd7f29 Add opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-02-26 17:44:24 +08:00
Xiangyu Wang
38266b8569 Lock channel at produce and consume
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-02-25 17:44:29 +08:00
Cai Yudong
84110d2684 Add tests/benchmark and tests/python_test using new python SDK
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2021-02-25 17:35:36 +08:00
bigsheeper
7cb28428ce Prevent consuming after loading collection
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-25 15:08:50 +08:00
cai.zhang
1d899513dd Unify service name in config file
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-02-24 18:46:19 +08:00
zhenshan.cao
0679954922 Refactor tso and global id allocator
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-24 17:12:06 +08:00
zhenshan.cao
7c5e2d89f6 Read node port from config
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-23 18:08:17 +08:00
dragondriver
b506754364 Add workflow of drop index in index service
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-02-23 11:57:18 +08:00
zhenshan.cao
bbfcbbdd68 Fix ci: paramstable and startup logic refactor
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-23 11:40:30 +08:00
sunby
ea1d9ea99a Replace log in dataservice
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-23 09:58:06 +08:00
yukun
e786ad6aa8 Fix inconsistency of produce msg and consume msg with rmq
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-02-22 16:34:15 +08:00
sunby
b07a371f25 Create a goroutine to receive msg for every consumer
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-22 09:58:34 +08:00
BossZou
7ac1821cf1 Add RocksMQ throughout test
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
2021-02-20 18:30:37 +08:00
yukun
346b0a9549 Add unittest for RmqTtMsgStream
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-02-20 15:58:19 +08:00
groot
af3736aff6 Fix rocksmq bug
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-02-18 14:13:55 +08:00
groot
833f1d49ab Let singlenode use rocksmq
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-02-09 15:57:10 +08:00
dragondriver
da94fbed0a Parse to map when the key of index params is named params
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-02-07 21:32:37 +08:00
yukun
563a8d66ae Add RmqTtMsgStream
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-02-07 15:10:13 +08:00
yukun
099220394e Add rmq_msgstream unittest and fix bugs
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-02-06 11:39:12 +08:00
godchen
c2ca2c276f Fix response check error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-02-04 19:34:35 +08:00
bigsheeper
c6c99ef32b Implement getSegmengInfo function
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-04 11:40:14 +08:00
Xiangyu Wang
fd9e42d3ed Update msgstream/unmarshal interface
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-02-03 17:30:10 +08:00
bigsheeper
abffdbff62 Remove partition tag
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 16:44:38 +08:00
yukun
35f6f4b7c2 Add consume in rmq_msgstream
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-02-03 15:18:54 +08:00
xige-16
4cd315408b Refactor query service’s meta interface
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-02-02 11:52:41 +08:00
Xiangyu Wang
99cef4b5c8 Add Distributed main entrypoint
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-01-29 17:08:31 +08:00
zhenshan.cao
ee0faddb07 Refactor proxy
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-29 09:27:26 +08:00
bigsheeper
4f914a2c30 Add docker file and main function of query node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-27 09:50:52 +08:00
neza2017
d1b23cb8c5 Add debug info
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-26 19:24:09 +08:00
sunby
f632ed0272 Fix data service init
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-26 18:45:12 +08:00
zhenshan.cao
e83ac41bca Add component for index
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-26 18:01:32 +08:00
xige-16
7c210310dc Add load function for query service
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-26 15:13:20 +08:00
XuanYang-cn
6c303c67b2 Rearrange update segment statistics of start position procedure
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-26 09:44:39 +08:00
cai.zhang
f940cc455a Add dockerfile for index
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-26 09:38:40 +08:00
FluorineDog
a8fd6e7686 Enable drop index/field_data and extract info visitor
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-25 18:59:44 +08:00
yukun
5600d06583 Add global_rmq
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-25 18:53:08 +08:00
XuanYang-cn
b794921b7e Add enum Role in internal.proto
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-23 18:56:08 +08:00
BossZou
01267776d2 Fix rocksmq consume invalid currendID
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
2021-01-23 10:15:57 +08:00
xige-16
511aa419fa Refactor query service
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-22 14:28:06 +08:00
neza2017
9828a52999 Add create index
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-21 10:01:29 +08:00
FluorineDog
7f044fff82 Enable search for sealed
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-21 15:29:52 +08:00
sunby
be6775671b Add error returned by Init,Start,Stop functions
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-19 17:38:01 +08:00
xige-16
76bc365181 Add seek function for pulsarTtMsgStream
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-20 17:34:50 +08:00
yukun
527c0c49df Add rocksmq unittest with goroutines
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-20 16:46:58 +08:00
BossZou
4588342fa1 Add IdAlocator service
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
2021-01-20 15:02:23 +08:00
sunby
3602376a10 Rename GetServiceStates to GetComponentStates and reconstruct NodeStates and ServiceStates
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-20 11:02:29 +08:00
cai.zhang
fb493cc235 Update interface name of GetIndexStates
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-19 18:32:57 +08:00
XuanYang-cn
6a13386393 Add DataNode package
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-19 11:37:16 +08:00
neza2017
1ce32d8775 Add master service
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 09:45:42 +08:00
yukun
b874a55c35 Add complex unittest for rocksmq
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-19 10:22:27 +08:00
zhenshan.cao
76d92e73d1 Replace old proto with new proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-18 21:39:56 +08:00
yukun
3aca35b10c Add rocksmq unittest
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-18 15:05:49 +08:00
bigsheeper
48821690b3 Rename Service interface to Component, and update segmentLoad strategy
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-18 10:38:41 +08:00
XuanYang-cn
37688559f1 Fix multi rowdata writenode crash bug
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-16 15:06:19 +08:00
yukun
44c75bdd2a Add consume function in rocksmq
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-15 20:12:26 +08:00
BossZou
31178d6a5c Add procude logic of RocksMQ
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
2021-01-15 16:06:35 +08:00
quicksilver
7fe061b846 Raneme init_devcontainer.sh to devcontainer.sh
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-15 14:38:36 +08:00
neza2017
56ab2bda2d Update proto:add partition id
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-15 11:53:22 +08:00
zhenshan.cao
63b21321d6 Refactor indexservice and update doc
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-15 13:50:27 +08:00
Xiangyu Wang
52bd6abdf8 Change RocksMQ interface
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-01-13 17:26:57 +08:00
GuoRentong
f3d2afa4f4 Update doc:detailed interface definition
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-11 18:35:54 +08:00
quicksilver
a6690dbc99 Fix syntax error: Bad fd number
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-11 15:25:19 +08:00
yukun
18cfd31d40 Fix Load and Remove with prefix
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-11 15:17:06 +08:00
sunby
5dfe9448ae Modify test_index
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-08 16:00:16 +08:00
GuoRentong
82ad488555 Update doc: rocksmq
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-08 18:37:24 +08:00
XuanYang-cn
65089ea362 Fix flush sync test bug and remove unecessary lock in meta_table
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-08 17:20:59 +08:00
GuoRentong
c4599e26de Update doc: node api
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-08 16:59:19 +08:00
bigsheeper
e13822043d Fix search test when partition not exists, add binary vector search validation
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-08 16:41:27 +08:00
godchen
fa73f17f94 Add opentracing doc
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-08 15:32:38 +08:00
dragondriver
ced56c84f8 Add support to build binary vector index
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-07 14:56:17 +08:00
godchen
8f1f118b26 Add query opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-07 14:24:35 +08:00
bigsheeper
a1e2ff3c26 Add gcNode to write node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-06 16:44:12 +08:00
neza2017
4015d7245d Merge operation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-06 14:45:50 +08:00
bigsheeper
810be533ab Add gcNode, and remove msg's downstream index
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-06 14:02:25 +08:00
zhenshan.cao
b06e01f523 Initmate querynode collection manager
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-06 11:31:36 +08:00
godchen
1c87faf8b3 Add insert request opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-06 11:17:35 +08:00
sunby
0a383608f3 Remove mock client in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-31 11:20:03 +08:00
FluorineDog
89f38e459a Fix bool type
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-04 17:52:42 +08:00
zhenshan.cao
6bd5702441 Add indexbuilder address to paramstable
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-04 14:53:20 +08:00
XuanYang-cn
c0797333ce Add ddl binlog metaTable in writenode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41 Add test for master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 16:19:42 +08:00