test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-04 21:09:06 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
10,216 Commits 96 Branches 105 Tags 186 MiB
169359448f
Commit Graph

61 Commits

Author SHA1 Message Date
congqixia
7ce7cb7a5e
Add a timed flush trigger mechanism (#10197) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-20 15:02:36 +08:00
XuanYang-cn
880056e664
[skip ci]Update license for datesync (#9959) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-15 18:01:38 +08:00
sunby
044b06421a
Use etcd to watch channel on datanode (#7115) 
issue: #6578
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-14 15:44:34 +08:00
congqixia
c90a97f95a
Fix Watch Channel canceled due to revision compacted (#9788) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-13 17:02:33 +08:00
XuanYang-cn
f0bedc7046
Add ut for data_node.go (#9489) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-11 18:50:30 +08:00
godchen
5e7648fc7c
Check data node test error (#9532) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-09 15:45:15 +08:00
godchen
937dbf4279
Add flushed segments (#9476) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-08 19:09:12 +08:00
XuanYang-cn
3be6672753
Add flush channel for delete_node (#8762) 
Resolves: #8761

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-09-28 18:22:16 +08:00
congqixia
d374ea71ba
Unify liveness check in datacoord and datanode (#8480) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-28 16:10:04 +08:00
groot
e4e762f2e8
Reduce go unittest time cost (#8503) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-09-26 12:50:24 +08:00
congqixia
2aa9deda1c
Improve logic when watch dm channel fails (#8386) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-23 18:31:55 +08:00
congqixia
6469d83072
Add datanode etcd alive check (#8363) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-23 14:29:54 +08:00
Xiaofan
f5173b595f
Add a channel prefix for all channels (#8166) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-23 10:53:53 +08:00
sunby
06cee85a39
Add data node unit tests (#7622) 
Issue: #6357
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-09-09 16:40:00 +08:00
dragondriver
89748ac639
Expose metrics of data cluster (#7177) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-01 10:13:15 +08:00
congqixia
881d0af1fc
Add sync logic in TestWatchChannel (#7188) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-08-20 17:48:13 +08:00
XuanYang-cn
86ca96a02e
Fix master branch unittest error (#7083) 
Resolves: #7080

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-08-13 14:00:14 +08:00
congqixia
e1b3273d1c
Add Datanode watch etcd channel (#6965) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-08-13 10:50:09 +08:00
XuanYang-cn
47df4a2e57
Fix DataNode BackGroudGC problem (#6551) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-08-03 09:59:24 +08:00
XuanYang-cn
dca6c6afde
Add unittest for BackGroundGC in DataNode (#6522) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-07-15 10:05:55 +08:00
XuanYang-cn
f0a68704e2
Add unittests for getChannelName in DataNode (#6346) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-07-08 15:52:40 +08:00
XuanYang-cn
441300140d
Tidy replica (#5920) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-21 16:00:22 +08:00
XuanYang-cn
ab7f642740
Add alias in paramtable (#5878) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-19 12:38:06 +08:00
sunby
33f5225968
Use physical channel to init msg stream (#5815) 
* Use physical channel to init msg stream

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix search collection error check (#1)

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: bigsheeper <954206947@qq.com>
 2021-06-18 16:02:05 +08:00
XuanYang-cn
18eb27aa7a
Tidy replica in datanode (#5805) 
- Remove not using functions
- Remove `initNode` logic in ParamTable
- Delete collection.go and collection_test.go

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-16 19:03:57 +08:00
XuanYang-cn
dea0a4e522 Idempotent of FlushSegment (#5746) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 16:36:36 +08:00
sunby
189ac881f3 Fix bugs (#5676) 
* Remove redundant session startup

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Register datanode after start success

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix meta snap shot

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix datanode message stream channel

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix bugs when drop empty collection

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix bug of getting pchan statistics from task scheduler

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix i/dist/dataservice test code

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix epoch lifetime not applied

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* fix datanode flowgraph dd node

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix handle datanode timetick bug

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix proxynode

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Apply extended seal policy

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add check for time tick

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix check

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix the repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the bug when send statistics of pchan

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the repack function when craete dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix bugs

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix describe collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix bug when send timestamp statistics

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add length check before flush request

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix SaveBinlog bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Add more log in datanode

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Put SegmentState.Flushing as the last one in enum to fit the client

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix params in GetInsertBinlogPaths

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Rename policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove unused ddl functions and fields

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Remove pchan when drop collection

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Add balanced assignment policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix master ut

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add lock in session manager

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for debug

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix some logic bug and typo

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix recover bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Get collection scheme of a specific timestamp

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Change CheckPoint to SegmentInfo in VchannelInfo

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Recover Unflushed segment numOfRows

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix dataservice unit tests

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 16:06:11 +08:00
XuanYang-cn
ac19711d74 Destroy DataNode when drop collections (#5638) 
* Destroy DataNode when drop collections

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* golanci-lint

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:04:48 +08:00
neza2017
0a82c6381f add auto flush for data node (#5609) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:04:48 +08:00
congqixia
d0d845805b Change WatchDmChannelsRequest proto (#5577) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 16:04:48 +08:00
XuanYang-cn
6c02ae4eb1 Remove Old msgposition logic (#5487) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 15:59:04 +08:00
XuanYang-cn
7a8b34ee9f Remove not using code in DataNode (#5476) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 15:59:04 +08:00
XuanYang-cn
55eac3ee7f Complete Flush procedure in DataNode (#5454) 
* Complete Flush proceduer in DataNode

Resolves: #5220

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix unittest

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 15:59:04 +08:00
sunby
c53afee616 DataNode scales flowgraph 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 15:59:04 +08:00
XuanYang-cn
fc630bc1c8
Correct auto-flush behaviour (#5282) 
Before this PR, DataNode considered auto-flush a valid flush
complete. It's wrong. So I open this PR to correct this behaviour
in DataNode.

Now binlog paths from auto-flush will be buffered in replica,
waiting until the manul flush to save into etcd all together.

See also: #5220, #5268
A follow up job of #5271 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-05-18 11:45:00 +00:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988) 
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-22 14:45:57 +08:00
Cai Yudong
51cae2c9ba
add datanode unittest to improve code coverage (#4982) 
add datanode unittest to improve code coverage 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-04-21 18:41:37 +08:00
XuanYang-cn
6deba1faa5 Add license in datanode package 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-04-19 15:16:33 +08:00
XuanYang-cn
91f5c48037 Fix datanode goroutine leak 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-30 09:47:27 +08:00
sunby
ca3187d173 Fix data race in datanode tests 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-24 16:35:38 +08:00
XuanYang-cn
51f669f1cb Flush segments concurrently & change etcd meta 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-23 18:50:13 +08:00
godchen
f3649f0419 Refactor interface and proto 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-12 14:22:09 +08:00
xige-16
09ae985daa Fix wrong error code in master_service_test.go 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-10 22:06:22 +08:00
neza2017
7b2d67242d Fix describe segment if index not exist 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-10 14:45:35 +08:00
quicksilver
455cc59dbb Update publish-test-images.yaml 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-09 13:55:35 +08:00
neza2017
2835bcdc4e Update masterservice unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-05 20:41:34 +08:00
godchen
7b9fdd7f29 Add opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-26 17:44:24 +08:00
XuanYang-cn
10374f15d7 Update log in datanode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-26 10:13:36 +08:00
Xiangyu Wang
fd9e42d3ed Update msgstream/unmarshal interface 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-03 17:30:10 +08:00
bigsheeper
abffdbff62 Remove partition tag 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 16:44:38 +08:00