Commit Graph

216 Commits

Author SHA1 Message Date
Cai Yudong
d5cd561449
Add metrics for master (#5515)
* metric

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* metric

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add MasterDDChannelTimeTick for metrics

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add metrics for all master grpc

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* register metrics

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add metrics register

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add comments for codacy check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix format issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-01 11:04:31 +08:00
Xiangyu Wang
ca0993c274 Update OWNERS
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-30 18:58:47 +08:00
yukun
c63524deb9
Add dablooms (#5440)
* Add dablooms cwrapper

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix undefined reference bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add cwrapper_dablooms_build shell script

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-28 10:23:30 +08:00
neza2017
f80dbab6ec
watch proxynode (#5424)
watch proxy node and,
send InvalidataCollectionMetaCache to each proxynode

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Resolves: #5423
2021-05-26 12:14:30 +00:00
yukun
a3fb1356de
Fix rocksmq seek error (#5432)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>

In milvus-standalone, when seek is called, rocksmq won't put a
mutex in MsgMutex channel, which cause that the consume
process hang. So the seek function in rocksmq needs to 
notify the consumer to consume.
2021-05-26 09:31:09 +00:00
godchen
200801271d
Fix GetSession error (#5401)
Fix GetSession error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-25 12:15:46 +00:00
godchen
042a061123
Fix session util error (#5370)
Fix session util error.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-24 08:31:06 +00:00
godchen
dd736ee8ba
Change session doc (#5355)
Change session doc

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-24 03:53:07 +00:00
zhenshan.cao
4e1b12269b
Fix bug: rand.Seed reset and generate duplicate number (#5361)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-05-24 02:50:37 +00:00
neza2017
e326eaad1e
master client,reconnect if grpc failed (#5360)
master client,reconnect if grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-24 01:44:49 +00:00
godchen
d189bf4d33
Change pulsar version (#5352)
Change pulsar version.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-22 03:36:49 +00:00
godchen
0eee7e5252
Change session (#5329)
Change session.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-21 11:28:52 +00:00
godchen
913712fbea
Improve session manager (#5310)
Improve session manager.
Issue #5174

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-20 07:07:25 +00:00
godchen
e224d1e725
Add SessionManager (#5288)
Add SessionManager.
Resolves: #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-19 10:36:05 +00:00
godchen
ccc0130bb4
Add exclusive parameter (#5278)
Add exclusive parameter.
Issue #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-18 11:07:27 +00:00
godchen
cca75583a6
Add session util (#5269)
Add session util.
Issue #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-18 03:39:21 +00:00
XuanYang-cn
9151e13480
Remove not using dependence and proto (#5226)
- FlushMsg proto message is no longer in use

- We don't depend on "github.com/prometheus/common/log"

Signed-off-by: yangxuan xuan.yang@zilliz.com
2021-05-14 08:50:55 +00:00
Cai Yudong
6f3a460b4e
Add pulsar seek example in pulsar_client_test.go (#5154)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-10 02:32:10 +00:00
FluorineDog
88f5642603
Add plan proto and support basic boolean expr parser (#5088)
**What type of PR is this?**
- [x] Feature

**What this PR does / why we need it:**
This PR supports boolean expression as DSL.
1. The goal of this PR is to support predicates
    like `A > 3 && not B < 5 or C in [1, 2, 3]`. 
2. Defines `plan.proto`, as Intermediate Representation (IR) 
    used between go and cpp. 
3. Support expr parser, convert predicate expr to IR
    in proxynode, while doing static check there
4. Support IR to AST in cpp, enable the execution
2021-04-29 08:48:06 +00:00
XuanYang-cn
cebdfb34f7
Enable multiple datanode (#5068)
Now it's able to start multiple datanode, however, they will process the same insert data.
Further changes are needed to truly enable multiple datanode:

While registering, dataservice need to return different insert channels for different datanode.

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-04-28 18:43:48 +08:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988)
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
sunby
fad94fb576
Remove unsed configs (#4990)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-04-22 03:04:59 +00:00
Xiangyu Wang
c731e2df78 Add license header in util
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-19 13:42:47 +08:00
XuanYang-cn
e281f6a4a1 Remove useless configs in config files
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-04-13 10:04:39 +08:00
godchen
deba964590 Change name of kvBase
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-12 18:09:28 +08:00
Cai Yudong
687d48b75e Improve code readibility
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2021-04-12 14:11:06 +08:00
yukun
964e7e0a28 Add unittest for mqclient
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-04-08 20:05:33 +08:00
dragondriver
decc80a525 Add unittest to proxy service
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-04-06 14:12:57 +08:00
Xiangyu Wang
87a1e0b662 Reorganize msgstream
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-02 13:48:25 +08:00
Xiangyu Wang
8b62790017 Reorganize mqclient
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-02 10:01:11 +08:00
dragondriver
6796c7bfe7 Add unittest to function util
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-03-31 18:22:17 +08:00
yukun
18e0581d5c Destroy readOptions after use
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-03-31 14:10:07 +08:00
yukun
604f96ed2b Replace consumers map by sync.map
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-03-31 10:05:24 +08:00
sunby
f8e879b423 Fix high cpu usage in singlenode
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-03-29 17:48:15 +08:00
sunby
ff8ebd5f42 Fix bug of estimating record size
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-03-29 15:14:51 +08:00
yukun
c4f9a6f22b Fix get position wrong
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-03-27 17:39:12 +08:00
xige-16
a48d46cfaf Alter msgID’s type to bytes
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-27 09:46:54 +08:00
yukun
432b53149b Refactor msgstream
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-03-26 20:10:11 +08:00
xige-16
7e60460aee close timeTickStream in flow graph when the collection is released
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-26 18:40:04 +08:00
neza2017
eca1e25470 Add msg id at master service
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-26 15:13:33 +08:00
dragondriver
41e1975611 Parallelize the processing of loading binlog and saving index files
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-03-26 11:19:02 +08:00
yukun
85bc2a54f0 Fix load twice hang bug
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-03-25 18:49:41 +08:00
quicksilver
7ad9b36207 Update reviewdog/action-hadolint github action version to v1.16.1
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-03-25 14:41:46 +08:00
bigsheeper
1634d75980 Release flowgraph properly
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-03-22 12:49:50 -05:00
紫晴
3be6dde459 Add smoke tag
Signed-off-by: 紫晴 <ting.wang@zilliz.com>
2021-03-23 14:35:30 +08:00
zhenshan.cao
c2734fa55f Fix bug and enchance system
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-03-22 16:36:10 +08:00
Xiangyu Wang
9ba9bf5e8a Release memory after rocksdb.write()
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-03-19 20:20:32 +08:00
yukun
0823382876 Refactor rmq_msgstream
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-03-19 19:33:21 +08:00
yukun
207f6ae773 Fix rocksmq CreateConsumerGroup
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-03-16 21:37:25 +08:00
XuanYang-cn
32660d547b Add endPositions in datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-03-16 17:55:42 +08:00