test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-03 04:19:18 +08:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity
7,119 Commits 95 Branches 105 Tags 184 MiB
0f4bd50ce3
Commit Graph

202 Commits

Author SHA1 Message Date
Cai Yudong
0f4bd50ce3 Support ETCD endpoints array (#5755) 2021-06-15 16:36:36 +08:00
neza2017
ac9dde7352 fix seek on pulsar msgstream (#5726) 
* fix seek on pulsar msgstream

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* test seek on pulsar msgstream

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* msg-seek test test

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add pure pulsar client seek test

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:14:29 +08:00
neza2017
351d87055a fix ttmsgstream (#5689) 
* fix msgstream

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix asconsumer

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:06:13 +08:00
sunby
189ac881f3 Fix bugs (#5676) 
* Remove redundant session startup

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Register datanode after start success

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix meta snap shot

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix datanode message stream channel

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix bugs when drop empty collection

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix bug of getting pchan statistics from task scheduler

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix i/dist/dataservice test code

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix epoch lifetime not applied

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* fix datanode flowgraph dd node

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix handle datanode timetick bug

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix proxynode

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Apply extended seal policy

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add check for time tick

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix check

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix the repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the bug when send statistics of pchan

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the repack function when craete dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix bugs

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix describe collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix bug when send timestamp statistics

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add length check before flush request

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix SaveBinlog bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Add more log in datanode

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Put SegmentState.Flushing as the last one in enum to fit the client

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix params in GetInsertBinlogPaths

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Rename policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove unused ddl functions and fields

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Remove pchan when drop collection

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Add balanced assignment policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix master ut

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add lock in session manager

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for debug

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix some logic bug and typo

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix recover bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Get collection scheme of a specific timestamp

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Change CheckPoint to SegmentInfo in VchannelInfo

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Recover Unflushed segment numOfRows

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix dataservice unit tests

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 16:06:11 +08:00
sunby
3662b3f0e5 Refactor dataservice (#5400) 
Pick up grpc calls to grpc_handler.go and refactor code format

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 15:59:01 +08:00
bigsheeper
cdbc6d2c94
Refactor query node and query service (#5751) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-15 12:41:40 +08:00
Cai Yudong
2ba93deb5e
Support master get segment info channel name from config file (#5606) 
* optimize msgstream unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* get data service segment info channel name from config file directly

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-04 16:28:34 +08:00
Xiangyu Wang
f791ccb246 Update OWNERS file 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-06-03 20:01:24 +08:00
yukun
dcb4161c9f
Fix retrieve_service (#5531) 
* Fix retrieve_service

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix retrieve_collection

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-02 19:18:33 +08:00
godchen
918458a1be
Add msgstream seek (#5483) 
* Add msgstream seek

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change test name

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove file

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* resolve conversation

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-31 17:46:30 +08:00
Cai Yudong
c468481a36
Optimize MqTtMsgStream and unittest (#5498) 
* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add getTsMsgFromConsumerMsg

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename some variables for better readability

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add chanWaitGroup

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add chanTtMsgTime

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename lastTimeStamp to currTimeStamp

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add comments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support unsynced ttMsg

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add TestStream_PulsarTtMsgStream_3

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove debug log

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* change ttMsg logic back

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix ci block issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix codacy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-31 11:31:30 +08:00
Xiangyu Wang
ca0993c274 Update OWNERS 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-05-30 18:58:47 +08:00
Cai Yudong
b414800d49
Update Seek interface (#5492) 
* update Seek

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update Seek for mqTtMsgStream

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* use Retry in Seek

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-29 23:21:34 +08:00
xige-16
fce792b8bf
Add historical and streaming module in querynode (#5469) 
* add historical and streaming

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix GetSegmentInfo

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* pass regression test

Signed-off-by: xige-16 <xi.ge@zilliz.com>

Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
 2021-05-28 10:26:30 +08:00
xige-16
3630eec92c
Fix receving wrong msg after seek (#5441) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-05-27 02:38:37 +00:00
zhenshan.cao
6766169878
Refactor repack logic for insertion (#5399) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-05-25 11:53:15 +00:00
zhenshan.cao
3e73775cc1
Add ComputeProduceChannelIndexes for msgstream (#5364) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-05-24 08:30:09 +00:00
yukun
f18dfb4ff6
Add RetrieveTask implementation (#5313) 
Resolves: #5257 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-05-20 07:02:31 +00:00
XuanYang-cn
9151e13480
Remove not using dependence and proto (#5226) 
- FlushMsg proto message is no longer in use

- We don't depend on "github.com/prometheus/common/log"

Signed-off-by: yangxuan xuan.yang@zilliz.com
 2021-05-14 08:50:55 +00:00
dragondriver
e5d4963ba0
Distinguish the dsl and expr in proxy and query node (#5118) 
Pass a parameter `IsExpr` from proxy to query node, so query 
node can create search plan according to the variable `IsExpr`.
It makes `segcore` unnecessary to distinguish `dsl` or `expr`
using dynamic exception. When `IsExpr` is set to true, query
node will call `CreatePlanByExpr` according to the passed
information about expression. Otherwise query node will keep
still to use `CreatePlan` according to the `dsl` information. At
the same time, this pr adds some unittests to `CreatePlanByExpr`,
these unittests translate already exist case with `dsl` to case with
`expr`.

Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-05-07 07:20:47 +00:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988) 
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-22 14:45:57 +08:00
xige-16
51b2b45e03 Add copyright for msgStream 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-04-19 11:30:19 +08:00
xige-16
78155d3959 Refactor load Release to async call in query service 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-04-15 15:15:46 +08:00
Cai Yudong
952f70af2a Clean makefile 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2021-04-09 14:07:03 +08:00
dragondriver
57831b9978 Add unittest to insertChannelsMap in proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-04-08 15:41:28 +08:00
dragondriver
bfb7ca0df6 Add unittest to time tick logic in proxy service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-04-07 10:06:17 +08:00
Xiangyu Wang
87a1e0b662 Reorganize msgstream 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-02 13:48:25 +08:00
Xiangyu Wang
8b62790017 Reorganize mqclient 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-02 10:01:11 +08:00
yukun
cdc96d1c32 Add unittest for msgstream_impl 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-04-01 17:29:21 +08:00
yukun
604f96ed2b Replace consumers map by sync.map 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-31 10:05:24 +08:00
xige-16
e823b56040 Add retry for seek function 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-30 13:53:49 +08:00
Xiangyu Wang
72917467ec Change singlenode logs output 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-30 10:52:42 +08:00
yukun
c4f9a6f22b Fix get position wrong 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-27 17:39:12 +08:00
XuanYang-cn
35656ea9ce Update pulsar client version and fix cannot close tcp connection error 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-27 13:58:52 +08:00
xige-16
3ab8b9af25 Dlete unlocked mutex in seek function 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-27 11:16:20 +08:00
xige-16
a48d46cfaf Alter msgID’s type to bytes 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-27 09:46:54 +08:00
yukun
432b53149b Refactor msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-26 20:10:11 +08:00
yukun
85bc2a54f0 Fix load twice hang bug 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-25 18:49:41 +08:00
quicksilver
7ad9b36207 Update reviewdog/action-hadolint github action version to v1.16.1 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-25 14:41:46 +08:00
zhenshan.cao
c2734fa55f Fix bug and enchance system 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-22 16:36:10 +08:00
groot
2280791128 Add memory message stream for search 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-03-19 20:16:04 +08:00
yukun
0823382876 Refactor rmq_msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-19 19:33:21 +08:00
yukun
207f6ae773 Fix rocksmq CreateConsumerGroup 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-16 21:37:25 +08:00
XuanYang-cn
32660d547b Add endPositions in datanode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-16 17:55:42 +08:00
Xiangyu Wang
fc6472eb08 Implement RocksMQ client 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-13 11:46:50 +08:00
godchen
f3649f0419 Refactor interface and proto 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-12 14:22:09 +08:00
xige-16
09ae985daa Fix wrong error code in master_service_test.go 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-10 22:06:22 +08:00
neza2017
7b2d67242d Fix describe segment if index not exist 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-10 14:45:35 +08:00
yukun
0d499a9f99 Fix search waiting for seconds in singlenode 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-09 17:42:13 +08:00
quicksilver
455cc59dbb Update publish-test-images.yaml 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-09 13:55:35 +08:00