Commit Graph

6878 Commits

Author SHA1 Message Date
dragondriver
f6ab3c81df
Add channelsMgr to proxy (#5323)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-05-21 01:52:20 +00:00
Xiangyu Wang
23c4de0eb8
Flush statistics for all int64 fields (#5318)
Resolves: #5262

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-20 10:38:45 +00:00
godchen
913712fbea
Improve session manager (#5310)
Improve session manager.
Issue #5174

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-20 07:07:25 +00:00
yukun
f18dfb4ff6
Add RetrieveTask implementation (#5313)
Resolves: #5257 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-20 07:02:31 +00:00
neza2017
c0daf8e41d
set timestamp on dd requests (#5298)
- set dd request's time tick on master
- send time tick into dd channel

See also: #5291

Signed-off-by: yefu.chen yefu.chen@zilliz.com
2021-05-20 06:14:14 +00:00
congqixia
38f5b0826f
Add SaveBinlogPaths grpc in dataservice (#5277)
* Add SaveBinlogPaths grpc in dataservice

* fix merged compile error

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-05-20 11:34:45 +08:00
ThreadDao
613cf21950
add cases for collection with data (#5305)
- add cases for collection with data
- cancel import *

See also: #5302 #5304 

Signed-off-by: ThreadDao yufen.zong@zilliz.com
2021-05-20 03:03:45 +00:00
XuanYang-cn
1c779f3efd
Refactor ddl binlog flush (#5303)
DDL won't auto-flush now

See also: #5289 , #5220 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-19 11:42:07 +00:00
yukun
a7bb701f73
Add Retrieve method grpc definition (#5294)
See also: #5253 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-19 10:45:15 +00:00
godchen
e224d1e725
Add SessionManager (#5288)
Add SessionManager.
Resolves: #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-19 10:36:05 +00:00
quicksilver
3db653f2d0
Update cpp thirdparty source URL (#5295)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

Resolves: #5293
2021-05-19 09:39:29 +00:00
congqixia
d5ab8ac345
Restore flush and stats stream pos (#5284)
Restore segment flush stream & statistic stream to last success pos

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-05-19 06:13:53 +00:00
XuanYang-cn
4b712284f2
Add ddl flush design (#5289)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-19 04:06:16 +00:00
ThreadDao
9b37cab922
Add cases for creating collections (#5286)
- add cases for creating collections
- change unique str from var to function
- improve coding standards

See also: #5273 #5285 

Signed-off-by: ThreadDao yufen.zong@zilliz.com
2021-05-19 03:34:52 +00:00
XuanYang-cn
fc630bc1c8
Correct auto-flush behaviour (#5282)
Before this PR, DataNode considered auto-flush a valid flush
complete. It's wrong. So I open this PR to correct this behaviour
in DataNode.

Now binlog paths from auto-flush will be buffered in replica,
waiting until the manul flush to save into etcd all together.

See also: #5220, #5268
A follow up job of #5271 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-18 11:45:00 +00:00
sunby
6dba0fd9cf
Add dataservice recover design docs (#5212)
This document includes changes about dataservice and new features 
to support the recovery after it crashed.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-05-18 11:20:26 +00:00
godchen
ccc0130bb4
Add exclusive parameter (#5278)
Add exclusive parameter.
Issue #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-18 11:07:27 +00:00
neza2017
d68ee3fbc2
query base on timeline (#5279)
query base on timeline, add timestamp on these queries:
- ShowCollections
- HasCollection
- DescribeCollection

Resolves: #5219

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-18 09:12:17 +00:00
XuanYang-cn
c3ac1375d8
Add buffer function for auto flush (#5271)
Auto-flush of a segment is not considered flush-completed.
So we need to buffer binlog paths generated by auto-flush.

See also: #5268 #5220 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-18 08:33:05 +00:00
neza2017
49f6542b1d
query master service's with timestamp (#5275)
let meta support snapshot
so collection could query meta with timestamp

Resolves: #5219

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-18 06:18:02 +00:00
Xiangyu Wang
c775744136
Add the design doc of retrieve entity (#5274)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-18 03:44:04 +00:00
godchen
cca75583a6
Add session util (#5269)
Add session util.
Issue #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-18 03:39:21 +00:00
XuanYang-cn
ca03488c83
Update design about O1-1 (#5268)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

The design procedure can be seen: #5220
2021-05-18 03:17:38 +00:00
Cai Yudong
e8ceb37f81
Add ShardsNum when create collection (#5245)
Updata PhysicalChannelNames and VirtualChannelNames based
on ShardsNum when create collection, and update collVChan2Chan
map.

Resolves: #5240 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-17 11:15:01 +00:00
ThreadDao
5d0aab05b9
add collection cases level and change get_connection to create_collection (#5249)
- update get_connection to create_connection
- add case level for collection
- optimize collection cases


Signed-off-by: ThreadDao yufen.zong@zilliz.com
2021-05-17 09:37:17 +00:00
quicksilver
689f750c1a
Fix run pytest bug (#5250)
* Change volumes with pytest docker-compose

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Fix run pytest bug

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-05-17 16:05:04 +08:00
quicksilver
16ad186874
Change volumes with pytest docker-compose (#5247)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-05-17 15:29:50 +08:00
ThreadDao
1dabbfc835
Add collection cases (#5239)
Add a part of the test case for the Collection interface

See also: #5224 #5231 

Signed-off-by: ThreadDao yufen.zong@zilliz.com
2021-05-17 02:14:35 +00:00
Cai Yudong
001795ee70
Optimize CreateIndex routine in master service (#5238)
Remove CreateIndexTask struct and indexTaskQueue

Resolves: #5228 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-15 10:08:08 +00:00
Xiangyu Wang
cd11c50932
[skip ci]Update mergify (#5237)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-15 13:02:31 +08:00
quicksilver
b37861ad5f
Increase ephemeral storage size for jenkins slave (#5235)
Signed-off-by: jeffoverflow zhifeng.zhang@zilliz.com
2021-05-15 02:13:17 +00:00
neza2017
a124568664
fix makefile (#5236)
just remove standalone and milvus from $GOPATH/bin,
not every thing, on command of make clean

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-15 02:02:48 +00:00
quicksilver
d17ed98fff
Allow jenkins slave pod to remain active for a while (#5232)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-05-14 14:09:40 +00:00
Cai Yudong
08bb1b2ec3
Resend unsuccessful ddMsg when master start (#5214)
Resend unsuccessful ddMsg when master start

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-14 21:26:06 +08:00
quicksilver
9ffb42d448
Deny jenkins slave pod to remain active for a while (#5225)
* Deny jenkins slave pod to remain active for a while

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-05-14 18:23:30 +08:00
quicksilver
0a21933d84
Specified Milvus Helm Chart path in E2E test (#5199)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-05-14 10:12:03 +00:00
godchen
168b917f5b
Add session util doc (#5229)
Add session util doc.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-14 09:09:14 +00:00
XuanYang-cn
9151e13480
Remove not using dependence and proto (#5226)
- FlushMsg proto message is no longer in use

- We don't depend on "github.com/prometheus/common/log"

Signed-off-by: yangxuan xuan.yang@zilliz.com
2021-05-14 08:50:55 +00:00
XuanYang-cn
aa9ed7e98f
add datanode design doc (#5206)
* add datanode design doc
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-14 14:56:32 +08:00
neza2017
b1e95ccf28
[skip-ci] master service recovery on power failure (#5216)
design doc about master service recover on power failure
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-14 06:25:46 +00:00
yukun
4e6e502df1
Add data sorter in storage (#5200)
GetEntityByID needs the flushed segment to be sorted by RowID field, 
then do binary search to get the target id and entities.

See also: #5177 

Signed-off-by: fishpenguin kun.yu@zilliz.com
2021-05-14 02:59:49 +00:00
godchen
b74afd7a0e
Add service registration (#5189)
Add service registration.
Part of Issue #5174.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-14 02:05:18 +00:00
紫晴
d35092cc09
Update check function (#5205)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2021-05-14 01:27:41 +00:00
紫晴
ec3d8c98a4
update test code (#5197)
update test code
2021-05-13 04:17:15 +00:00
quicksilver
8b7181c584
Update install_milvus.sh (#5188)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

Delete milvus PVC does not affect E2E Test workflow
2021-05-13 01:47:26 +00:00
Cai Yudong
7b83b11e95
Save ddmsg info and type into ETCD (#5173)
Save ddmsg info and type into ETCD to support re-send ddmsg 
when system restart or recover from failure (#5172)

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-12 07:33:53 +00:00
Xiangyu Wang
d543e2c6cc
[skip ci]Update mergify (#5180)
Signed-off-by: scsven <wxyucs@gmail.com>
2021-05-12 10:36:00 +08:00
yanliang567
acc631d632
[skip ci] Change code owner for test folders (#5178)
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2021-05-12 09:42:26 +08:00
quicksilver
9e848b184e
Add publish-krte-images.yaml (#5179)
* Add publish-krte-images.yaml

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update publish-krte-images.yaml

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-05-11 19:28:29 +08:00
紫晴
1c8717c2a3
add refactor test codes (#5171)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2021-05-11 17:59:29 +08:00