test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-05 05:18:52 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
20,002 Commits 96 Branches 105 Tags 186 MiB
c344083f22
Commit Graph

46 Commits

Author SHA1 Message Date
jaime
6423b6c718
enhance: move rocksmq from internal to pkg (#34165) 
pr:  https://github.com/milvus-io/milvus/pull/33881
issue:  https://github.com/milvus-io/milvus/issues/33956

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-26 13:36:05 +08:00
yihao.dai
b1e74dc7cb
enhance: [cherry-pick] Decouple compaction from shard (#34157) 
This PR cherry-picks the following commits:

- Implement task limit control logic in datanode.
https://github.com/milvus-io/milvus/pull/32881
- Load bf from storage instead of memory during L0 compaction.
https://github.com/milvus-io/milvus/pull/32913
- Remove dependencies on shards (e.g. SyncSegments, injection).
https://github.com/milvus-io/milvus/pull/33138
- Rename Compaction interface to CompactionV2.
https://github.com/milvus-io/milvus/pull/33858
- Remove the unused residual compaction logic.
https://github.com/milvus-io/milvus/pull/33932

issue: https://github.com/milvus-io/milvus/issues/32809

pr: https://github.com/milvus-io/milvus/pull/32881,
https://github.com/milvus-io/milvus/pull/32913,
https://github.com/milvus-io/milvus/pull/33138,
https://github.com/milvus-io/milvus/pull/33858,
https://github.com/milvus-io/milvus/pull/33932

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-25 20:22:03 +08:00
yiwangdr
b1eacb2ae8
feat: datacoord/node watch based on rpc (#32036) 
issue: https://github.com/milvus-io/milvus/issues/25309

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-05-07 15:49:30 +08:00
yihao.dai
0fe5e90e8b
enhance: Remove import v1 (#31403) 
Remove all code and logic related to import v1.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-22 15:29:09 +08:00
XuanYang-cn
ae180d1628
enhance: Change ChannelManager to interface (#29300) 
Rewrite cluster test
issue: #28854

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-25 19:24:46 +08:00
aoiasd
3c32ba2407
enhance: pack datacoord Cluster and SessionManager with interface and mock them (#28869) 
relate: https://github.com/milvus-io/milvus/issues/28861
https://github.com/milvus-io/milvus/issues/28854

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-11 17:52:37 +08:00
XuanYang-cn
a153950b10
Change channel to Interface (#27839) 
This PR changes `*channel` into RWChannel interface

See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-13 11:16:18 +08:00
Xiaofan
2ea7579dbb
Reduce rpc size for GetRecoveryInfoV2 (#27483) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-23 21:44:09 +08:00
congqixia
49516d44b4
Add ctx parameter and log tracer for watch and selectNodes (#27809) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-20 04:22:11 +08:00
jaime
e386a62fae
Remove recollect segment stats during starting datacoord (#27410) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-16 10:26:09 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface (#27186) 
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-09-26 09:57:25 +08:00
SimFG
26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
yihao.dai
63b86b32a6
Add server id validation interceptor (#26395) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-08-17 20:20:20 +08:00
congqixia
0727e017c4
Add Promethues suite test utility (#26046) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-02 10:05:05 +08:00
congqixia
7f51655ce3
Add unit test for DataNodeNum metrics value (#25985) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-31 16:59:03 +08:00
Bingyi Sun
268368031c
Remove redundant kv mocks (#25420) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-07-07 19:38:26 +08:00
yiwangdr
c7b851f870
add interface for non-watch metakv (#25092) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-06-26 09:20:44 +08:00
yah01
ebd0279d3f
Check error by Error() and NoError() for better report message (#24736) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-06-08 15:36:36 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
Xiaofan
3d6bded115
Fix Row Number Mismatch (#22307) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-27 10:41:46 +08:00
Enwei Jiao
697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
Enwei Jiao
89b810a4db
Refactor all params into ParamItem (#20987) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-07 18:01:19 +08:00
bigsheeper
cd19d99ad7
Add channel level checkpoint (#20350) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-10 22:13:04 +08:00
Enwei Jiao
956c5e1b9d
Make Params singleton (#20088) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-04 14:25:38 +08:00
congqixia
2e6ddd7f2a
Add datanode&channel match check for Flush (#19985) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-22 12:09:28 +08:00
Ten Thousand Leaves
b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-09-26 18:06:54 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-16 16:56:49 +08:00
Jiquan Long
641afae22f
Watch channels with start positions (#18744) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-08-22 14:42:52 +08:00
Ten Thousand Leaves
f5e63177d2
Fix an issue with bulk load with multiple DataNodes (#17324) 
issue: #17300
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-06-02 18:54:04 +08:00
Ten Thousand Leaves
5e1e7a6896
Fix an issue that causes flush hanging (#17169) 
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-25 14:34:00 +08:00
Ten Thousand Leaves
45be3deb3f
Implement RC, DC, DN calling path for import. (#16321) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-01 11:33:28 +08:00
XuanYang-cn
a77dd1034c
Adopt the extended state in DataCoord (#16200) 
ChannelWatchInfo with ToWatch, ToRelease will trigger a timer.
ChannelManager now reacts to different ChannelWatch states.

- WatchSuccess > log this info

- WatchFailure/WatchTimeout > ToRelease

- ReleaseSuccess > Delete, reassign if not from DropCollection

- ReleaseFailure/ReleaseTimeout > Cleanup subscription and Delete,
  reassgin if not from DropCollection.

Some Notes:
1. Reassignment will add this channel to buffer if there's only one node.

See also: #15846

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-28 22:33:27 +08:00
Nemo
3eb074a9b0
Update some comments for data coord with some minor code refactoring. (#15686) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-02-22 13:15:51 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
Bingyi Sun
175a656ff1
Unwatch dropped channel when init channel manager (#11986) 
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-17 23:25:12 +08:00
sunby
7fbb469081
Change channel's field to upper case to log details (#10567) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-26 20:15:41 +08:00
XuanYang-cn
102bb36440
[skip ci]Update license for dc cluster (#10593) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-25 20:23:16 +08:00
godchen
c2fa2e940d
Check cluster error (#10116) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-21 10:57:03 +08:00
congqixia
7ce7cb7a5e
Add a timed flush trigger mechanism (#10197) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-20 15:02:36 +08:00
sunby
044b06421a
Use etcd to watch channel on datanode (#7115) 
issue: #6578
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-14 15:44:34 +08:00
congqixia
d3be316bc2
Add Comment for datacoord cluster (#7676) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-10 11:26:00 +08:00
congqixia
1f590d6c26
Add datacoord services related unit tests (#7467) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-06 11:12:42 +08:00
sunby
ee38d14f5d
Fix channel lost after data node crash (#6545) 
If we start up 2 data nodes and one of them crashes. We expect that all
channels of crashed node will be allcoated to the alive node. But now we
discover that these channels are lost after data node crash. The reason
is we pass a NodeInfo with empty channel info. We fix it and improve log
print.

issue: #6501
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-15 16:38:31 +08:00
sunby
a8e5fd2024
Refactor cluster in dataservice (#6356) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-12 11:03:52 +08:00
congqixia
580e3a57cf
stop add new node into cluster before validated (#6241) 
* stop add new new into cluster

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix startup & test cases

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-07-01 14:32:16 +08:00
sunby
c863a193de
Rename filename from dataservice to datacoordinator (#5946) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-22 10:42:07 +08:00