test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-03 04:19:18 +08:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity
16,037 Commits 95 Branches 105 Tags 184 MiB
2b02b869ec
Commit Graph

130 Commits

Author SHA1 Message Date
xiyichan
ca129d4308
Support configurable msgstream (#16131) 
Signed-off-by: xiyichan <2863768433@qq.com>
 2022-03-24 10:15:25 +08:00
groot
1c4b949a1d
modify import rpc interface (#16129) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-03-22 15:11:24 +08:00
groot
2078b24dca
Prepare for import (#16083) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-03-21 15:47:23 +08:00
jaime
29975a7a26
Remove Consume() API from mqstream interface (#15886) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-11 20:09:59 +08:00
Cai Yudong
3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-04 11:17:56 +08:00
jaime
7a3a721380
Reconstruct mqstream module (#15784) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-03 21:57:56 +08:00
congqixia
b0923f1299
Fix DataCoord panics if message queue service quits before it (#15702) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-23 11:09:52 +08:00
congqixia
c7f55c2e34
Make SessionWatch keep watch even no Rewatch func when find ErrCompacted (#15497) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-15 15:07:48 +08:00
Cai Yudong
f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-07 10:09:45 +08:00
Cai Yudong
4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-02 00:35:43 +08:00
xige-16
cc5bee4684
Stop handoff if segment has been compacted (#15181) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-13 21:49:35 +08:00
Cai Yudong
7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-09 23:45:33 +08:00
Cai Yudong
f8f61fb116
Use BaseParams etcd configuration for all components (#15012) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-07 16:15:26 +08:00
Xiaofan
69087ff8dd
Support embedded etcd (#14333) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 14:35:21 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
Xiangyu Wang
8960f54454
Improve import statement (#13731) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-20 15:47:38 +08:00
congqixia
1dcd06cc78
Add log file info for all kind of segment binlogs (#13606) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-19 20:00:42 +08:00
cai.zhang
9f23fc7f2a
Register the service when the component state is healthy (#13248) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-15 11:47:10 +08:00
Bingyi Sun
0c39e0370c
Move segment statistics update to datanode tt loop (#13233) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-15 10:53:16 +08:00
Xiangyu Wang
405b3cd932
Use the same lint rules with golint in revive (#13324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-14 15:31:07 +08:00
XuanYang-cn
a9a332dbcf
Fix empty segment compact and load (#12710) 
1. Fix compaction save empty segment bug
2. Fix load empty segment bug
3. Add UT for dmlchannels to 100%

Resolves: #12450

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-09 11:03:08 +08:00
congqixia
22fddc537c
Fix not all segment state is drop after DropVirtualChannel (#12702) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-03 18:55:34 +08:00
congqixia
8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode (#12563) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-02 16:39:33 +08:00
congqixia
06ced4d1d1
Add DropVirtualChannel for DataCoord (#12361) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-29 22:35:41 +08:00
congqixia
eb48013042
Set start position for compact result segment (#12280) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-25 17:19:17 +08:00
congqixia
2ee0c7c074
Add compacted segment into check list when drop collection (#12272) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-25 15:13:15 +08:00
bigsheeper
e245ba87a9
Add excluded segments for dropped segemnts (#12247) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-11-25 14:25:15 +08:00
congqixia
8e0c8febe9
Add dropped segment ids in vchaninfo (#12230) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-23 18:59:15 +08:00
Bingyi Sun
01be91116e
Add GetFlushState interface (#12170) 
issue: #12075
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-23 10:55:14 +08:00
dragondriver
ee0f753f7a
Fix datarace between GetComponentStates and Register (#11935) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-11-19 13:57:12 +08:00
congqixia
dfc70108c4
Add load collection for AssignSegmentID (#12077) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-18 22:31:34 +08:00
Bingyi Sun
175a656ff1
Unwatch dropped channel when init channel manager (#11986) 
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-17 23:25:12 +08:00
Bingyi Sun
e3e4dc7153
Improve GetVChanPosistion and fix unit tests (#11888) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-16 21:41:50 +08:00
congqixia
d38043f235
Filter with partition id in GetVChanPosition (#11746) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-13 08:45:09 +08:00
bigsheeper
93149c5ad9
Load growing segment in query node (#11664) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-11-12 18:27:10 +08:00
Bingyi Sun
5c8978e5d6
Fix drop segment bug (#11709) 
issue: #11708
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-12 17:31:11 +08:00
Bingyi Sun
54b40da4c5
Add segment drop logic (#11621) 
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-12 00:22:42 +08:00
Bingyi Sun
c3f0c5a3ff
Add compaction interfaces in proxy (#11431) 
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-09 14:47:02 +08:00
sunby
c26cd79532
Add compaction (#9906) 
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-11-05 22:25:00 +08:00
congqixia
390fad4eb7
Fix LatestPosition option conflict with earliest patch (#10907) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-29 21:30:49 +08:00
congqixia
c019e80dbb
Change SeekPosition to earliest of all segments (#10771) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-28 14:30:26 +08:00
godchen
095a2378ca
Add datanode ut (#10433) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-25 22:52:27 +08:00
XuanYang-cn
548634ff05
[skip ci]Update license for dc segmentinfo (#10588) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-25 19:44:37 +08:00
congqixia
285c3f63e3
Trim segmentinfo binlog for VChaninfo usage (#10425) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-22 14:35:13 +08:00
godchen
3f713a1104
Check server test error (#9954) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-15 17:12:35 +08:00
sunby
bfc9e47e2c
Fix data race in datacoord test (#9936) 
issue: #9935
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-15 15:56:35 +08:00
congqixia
6b8ff60de3
Close event channel when watch event found error (#9819) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-14 19:20:35 +08:00
sunby
044b06421a
Use etcd to watch channel on datanode (#7115) 
issue: #6578
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-14 15:44:34 +08:00
congqixia
02dc2a2453
Add GetVChanPosition new cases ut (#9544) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-09 20:57:33 +08:00
godchen
937dbf4279
Add flushed segments (#9476) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-08 19:09:12 +08:00
godchen
2792f85211
Handle server error (#9032) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-03 22:17:56 +08:00
sunby
6f08430cb8
Check error in server test (#9012) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-01 21:00:46 +08:00
sunby
332025aace
Check error in server_test.go (#9011) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-01 20:58:10 +08:00
sunby
2ea7c0af25
Check error in datacoord server test (#8802) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-09-28 22:04:06 +08:00
sunby
eee075b10e
Check error in server test (#8801) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-09-28 21:54:14 +08:00
Xiaofan
f5173b595f
Add a channel prefix for all channels (#8166) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-23 10:53:53 +08:00
congqixia
21f15d66bb
Add option for DataNode creation (#8240) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-18 19:10:07 +08:00
Xiangyu Wang
42aafdc407
Remove unused msg from msg.go (#7612) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-09-09 19:56:00 +08:00
sunby
49b6c3101d
Remove datacoord useless code (#7673) 
Issue: #7624
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-09-09 19:08:03 +08:00
congqixia
fb4e23bc79
Add datacoord server unit tests (#7499) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-06 17:02:41 +08:00
congqixia
1f590d6c26
Add datacoord services related unit tests (#7467) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-06 11:12:42 +08:00
dragondriver
89748ac639
Expose metrics of data cluster (#7177) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-01 10:13:15 +08:00
Xiaofan
58b2089692
Import Embedded Etcd Kv Part 1(#7112) (#7174) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-08-24 09:45:51 +08:00
congqixia
be5c492e5a
Add datacoord unit tests (#7197) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-08-20 17:50:12 +08:00
sunby
4f7c3e7d3a
Save binlog path in SegmentInfo (#6590) 
issue: #6589
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-08-19 13:00:12 +08:00
sunby
9551dc3b7e
Fix segment auto flush bug (#7085) 
issue: #7084
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-08-16 11:00:09 +08:00
sunby
454620dfaa
Add unit test in datacoord (#6985) 
issue: #6984
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-08-04 16:59:25 +08:00
sunby
fc74a0f578
Auto create new segments when allocating rows more than (#6665) 
max number of rows per segment

If user insert too much rows in a request. Now we will return a failed
response. Maybe auto creating new segments to hold that much rows is a
better way.

issue: #6664
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-23 21:58:33 +08:00
sunby
63387c83ef
Remove collection check when saving binlog (#6658) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-23 21:23:22 +08:00
sunby
83be910baa
Refactor segment manager (#6441) 
issue: #6440
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-12 17:24:25 +08:00
sunby
a8e5fd2024
Refactor cluster in dataservice (#6356) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-12 11:03:52 +08:00
sunby
017484e093
Refactor meta in data coord (#6300) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-07 14:02:01 +08:00
sunby
07b0989628
Add GetFlushedSegments in data coordinator (#6253) 
Send segment flush message by grpc and add a new interface
`GetFlushedSegments`

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-02 11:16:20 +08:00
congqixia
ae072b4f1d
Add candidate management for datacoord dn cluster (#6196) 
* Add candidate management for datacoord dn cluster

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix test cases

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-30 10:20:15 +08:00
godchen
3785bb6330
Change retry times (#6083) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-24 19:05:06 +08:00
sunby
46f86a0354
Return sealed segments when calling flush (#6011) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-23 16:56:11 +08:00
godchen
99be4c09a5
change retry (#5996) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-23 09:24:10 +08:00
sunby
0978b93a21
Refactor data coordinator (#5982) 
Rename variable name and make error msg more clear

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-22 18:24:08 +08:00
sunby
d2c8462dac
Rename params in data_coord.yaml (#5968) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-22 16:18:21 +08:00
sunby
c863a193de
Rename filename from dataservice to datacoordinator (#5946) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-22 10:42:07 +08:00