Commit Graph

100 Commits

Author SHA1 Message Date
XuanYang-cn
3cd0584a04
Add compaction in datanode (#11353)
See also: #9530

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-08 19:49:07 +08:00
bigsheeper
5c58924420
Add cgroups to get container memory and check index memory in segment loader (#10363)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-22 20:49:10 +08:00
sunby
044b06421a
Use etcd to watch channel on datanode (#7115)
issue: #6578
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-14 15:44:34 +08:00
JackLCL
41b4264ac2
[skip ci]Fix doc error in milvus_create_index_en.md (#9653)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-10-11 19:06:38 +08:00
congqixia
619beb0d2c
Upgrade pulsar-client-go to BUGFIX commit (#8835)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-29 15:02:00 +08:00
XuanYang-cn
ff8fef6ad0
Upgrade pulsar-client-go to 0.6.0 (#7909)
MessageID interface of pulsar-client-go adds
4 more methods. So I add these methods into
`MessageID` interface.

Resolves: #7770

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-15 18:13:49 +08:00
Xiaofan
619c4393f3
Refine logs in proxy search path(#7357) (#7753)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-09-12 17:54:01 +08:00
dragondriver
960741ee16
Support more detailed metrics information (#7472)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-03 19:35:47 +08:00
Xiaofan
58b2089692
Import Embedded Etcd Kv Part 1(#7112) (#7174)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-08-24 09:45:51 +08:00
godchen
db25550fd0
Add segment filter interface (#7231)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-23 17:49:50 +08:00
dragondriver
ad77a6e0d4
Fix #6859, increase the MaxCallRecvMsgSize and MaxCallSendMsgSize of grpc client (#6861)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-04 13:03:24 +08:00
godchen
b9f5949680
Add vector chunk manager (#6613)
* add file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add vector file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* rename

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix ut error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix memory error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-20 15:06:09 +08:00
dragondriver
676a30e418
Use channelsMgr to manage search message stream (#5819)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-18 10:33:58 +08:00
godchen
b7bf26b486
Update grpc trace (#5817)
* Update grpc trace

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove useless code in mod

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-17 14:17:56 +08:00
dragondriver
e76830ab09
Support column-based insert request in proxy (#5560)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-03 15:03:34 +08:00
zhenshan.cao
6766169878
Refactor repack logic for insertion (#5399)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-05-25 11:53:15 +00:00
zhenshan.cao
3e73775cc1
Add ComputeProduceChannelIndexes for msgstream (#5364)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-05-24 08:30:09 +00:00
godchen
d189bf4d33
Change pulsar version (#5352)
Change pulsar version.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-22 03:36:49 +00:00
zhenshan.cao
70241a8bf9
Cancel local timestamp caching logic (#5327)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-05-21 05:11:21 +00:00
godchen
895037889e
Upgrade grpc version from 1.26 to 1.27 (#5128)
Upgrade grpc version from 1.26 to 1.27
Signed-off-by: godchen qingxiang.chen@zilliz.com
2021-05-07 18:30:11 +08:00
FluorineDog
88f5642603
Add plan proto and support basic boolean expr parser (#5088)
**What type of PR is this?**
- [x] Feature

**What this PR does / why we need it:**
This PR supports boolean expression as DSL.
1. The goal of this PR is to support predicates
    like `A > 3 && not B < 5 or C in [1, 2, 3]`. 
2. Defines `plan.proto`, as Intermediate Representation (IR) 
    used between go and cpp. 
3. Support expr parser, convert predicate expr to IR
    in proxynode, while doing static check there
4. Support IR to AST in cpp, enable the execution
2021-04-29 08:48:06 +00:00
godchen
139b334732
Add kv etcd unittest and remove useless code (#4974)
* Add kv etcd unittest and remove useless code

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-21 18:45:35 +08:00
neza2017
56d367b1e9 Add unit test for master service
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-04-09 16:10:12 +08:00
sunby
1d7195e036 Add minio and etcd stats watcher
Our system needs to watch the performance of etcd and minio. For now I just add the creation stats.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-04-08 14:14:43 +08:00
XuanYang-cn
35656ea9ce Update pulsar client version and fix cannot close tcp connection error
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-03-27 13:58:52 +08:00
xige-16
7e60460aee close timeTickStream in flow graph when the collection is released
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-26 18:40:04 +08:00
zhenshan.cao
c2734fa55f Fix bug and enchance system
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-03-22 16:36:10 +08:00
yukun
0823382876 Refactor rmq_msgstream
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-03-19 19:33:21 +08:00
dragondriver
af32f442bb Split big insert message into serveral smaller
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-03-09 16:18:58 +08:00
sunby
aeb3038545 Remove GetCount in data service
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-03-08 10:49:44 +08:00
Xiangyu Wang
e5d595564a Add log for singlenode
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-03-05 18:16:50 +08:00
sunby
ddddd65d10 Delete internal/errors package
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-03-05 10:15:27 +08:00
godchen
7b9fdd7f29 Add opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-02-26 17:44:24 +08:00
zhenshan.cao
bbfcbbdd68 Fix ci: paramstable and startup logic refactor
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-23 11:40:30 +08:00
sunby
ea1d9ea99a Replace log in dataservice
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-23 09:58:06 +08:00
sunby
b07a371f25 Create a goroutine to receive msg for every consumer
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-22 09:58:34 +08:00
neza2017
1d3fdebcf4 Add drop index for proxy
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-20 15:38:44 +08:00
sunby
5769c000a1 Find failed python tests and add skip mark "r0.3-test"
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-19 15:37:04 +08:00
bigsheeper
d8f41db823 Fix search crash error
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-19 13:56:01 +08:00
sunby
5b61182140 Add zap log
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-18 19:53:55 +08:00
Xiangyu Wang
99cef4b5c8 Add Distributed main entrypoint
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-01-29 17:08:31 +08:00
dragondriver
eba6bb288c Refactor the sructure of proto file
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-22 09:36:18 +08:00
xige-16
d22c2cd203 Add proto for query service loading collection
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-18 19:32:08 +08:00
GuoRentong
f3d2afa4f4 Update doc:detailed interface definition
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-11 18:35:54 +08:00
quicksilver
a6690dbc99 Fix syntax error: Bad fd number
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-11 15:25:19 +08:00
yukun
18cfd31d40 Fix Load and Remove with prefix
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-11 15:17:06 +08:00
sunby
5dfe9448ae Modify test_index
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-08 16:00:16 +08:00
yukun
d11f00cd9f Add rocksdb in kv
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-08 19:07:57 +08:00
GuoRentong
82ad488555 Update doc: rocksmq
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-08 18:37:24 +08:00
XuanYang-cn
65089ea362 Fix flush sync test bug and remove unecessary lock in meta_table
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-08 17:20:59 +08:00