test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-04 04:49:08 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
18,728 Commits 96 Branches 105 Tags 185 MiB
5474bce9d2
Commit Graph

895 Commits

Author SHA1 Message Date
XuanYang-cn
623939c9f5
enhance: Remove not in use policies (#29448) 
THe results don't meet our requirements, and the code hasn't been
maintained for a long time.

See also: #29447

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-28 10:38:46 +08:00
wei liu
839a72129e
fix: Auto balance param can't be updated by dynamic (#29501) 
This PR fixed that auto balance param can't be updated by dynamic

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-27 14:30:53 +08:00
XuanYang-cn
ae180d1628
enhance: Change ChannelManager to interface (#29300) 
Rewrite cluster test
issue: #28854

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-25 19:24:46 +08:00
SimFG
dd9c61831d
enhance: Support to get the param value in the runtime (#29297) 
/kind improvement
issue: #29299

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-12-22 18:36:44 +08:00
SimFG
67ab0e424b
fix: Clean the compaction plan info to avoid the object leak (#29365) 
issue: #29296

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-12-22 12:00:43 +08:00
yah01
a0e1a1eb31
feat: support enable/disable mmap for index (#29005) 
support enable/disable mmap for index, the user could alter the index's
mode by `AlterIndex` method
related: https://github.com/milvus-io/milvus/issues/21866

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-21 18:07:24 +08:00
cai.zhang
1b4d2674b3
fix: Set the default index name to the name of the existing index (#29275) 
issue: #29269

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-20 17:22:41 +08:00
aoiasd
010d8362ad
fix: datanode L0 segment num wrong (#29050) 
relate:  https://github.com/milvus-io/milvus/issues/27675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-20 11:20:44 +08:00
congqixia
cbf0f9c527
enhance: change cp metric to absolute unix ts (#29328) 
See also #29327

Change channel checkpoint metrics to unix seconds instead of checkpoint
timestamp lag value

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-20 11:04:45 +08:00
congqixia
8a63e53421
enhance: Add http method to control datacoord garbage collection (#29052) 
See also #29051

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 19:26:39 +08:00
XuanYang-cn
cc727ace61
fix: Set compacted segments' level to level one (#29190) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-14 18:46:40 +08:00
yah01
58dbb7872a
fix: forbid balancing level zero segments (#29168) 
related #29128
Signed-off-by: yah01 <yah2er0ne@outlook.com>

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-14 14:30:39 +08:00
XuanYang-cn
2c62bcbbf6
fix: Fix some left comments (#29027) 
See also: #28814

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-13 18:48:39 +08:00
Bingyi Sun
ad866d2889
feat: integrate storagev2 into index build process (#28995) 
issue: https://github.com/milvus-io/milvus/issues/28994

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-13 17:24:38 +08:00
wei liu
69a71e11d1
enhance: Add config for auto upgrade segment index (#29112) 
This PR add config for auto upgrade segment index to index engine's
version.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-12 11:56:37 +08:00
SimFG
ef18e6a532
enhance: Use a non-blocking method to trigger compaction when saveBinlogPath is executed (#28941) 
/kind improvement
issue: #28924

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-12-12 10:16:39 +08:00
aoiasd
3c32ba2407
enhance: pack datacoord Cluster and SessionManager with interface and mock them (#28869) 
relate: https://github.com/milvus-io/milvus/issues/28861
https://github.com/milvus-io/milvus/issues/28854

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-11 17:52:37 +08:00
XuanYang-cn
5bac7f7897
fix: Fix L0 compaction in datacoord (#28814) 
See also: #27606

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-05 18:44:37 +08:00
yihao.dai
d26b563a8b
feat: Define import API and metadata (#28731) 
Define the new rpc and metadata for ImportV2.

see also: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-04 19:56:35 +08:00
congqixia
038eebba4d
enhance: Move datacoord broker into separate package (#28876) 
See also #28861

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-01 10:22:34 +08:00
smellthemoon
adce6aab4f
fix:handle err when ManualCompaction (#28804) 
handle err when ManualCompaction
#28644

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-11-30 17:06:31 +08:00
wayblink
b592f08db3
enhance: Use smaller lock in datacoord.meta.UpdateChannelCheckpoint (#28753) 
/kind enhancement

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-11-30 11:00:28 +08:00
congqixia
3f9da2b8a0
fix: Check whether compaction plan exist before update (#28826) 
Fix: #28825

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-29 18:54:27 +08:00
XuanYang-cn
321c5c32e3
fix: Separate schedule and check results loop (#28692) 
This PR:

- Separates compaction scheduler and check results loop So that slow in
check-loop doesn't influence execution.

- Cleans compaction tasks when drop a vchannel so dropped-channel's
compaction tasks won't be checked over and over again.

  - Skips meta change when meta's already changed, avoid panic
  - Remove not inuse injectDone(bool) parameter

See also: #28628, #28209

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-29 10:50:29 +08:00
XuanYang-cn
606ec77b66
enhance: Unify levelzero segment config in DN (#28720) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-28 18:04:26 +08:00
congqixia
5025c10e8b
enhance: Remove channel cp lag metrics when DropChannel (#28784) 
See also #28765
Remove metric when Drop channel grpc execute succeed

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-28 14:24:25 +08:00
congqixia
dd682658da
enhance: Add channel checkpoint ts lag metric (#28767) 
See also #28765

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-28 07:00:28 +08:00
jaime
b1e0a27f31
enhance: Add logs for each step during service initialization (#28624) 
/kind improvement

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-11-27 16:30:26 +08:00
aoiasd
8a4cfb7d6a
enhance: add l0 metric and fix datacoord no need drop l0 segment when flush (#28373) 
relate: https://github.com/milvus-io/milvus/issues/27675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-24 15:58:24 +08:00
smellthemoon
79c0edb1d8
enhance:Remove msgbase unnecessary assignments (#28511) 
remove some unnecessary assignments, for the reason that
commonpbutil.NewMsgBase has default value.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-11-24 15:02:39 +08:00
congqixia
a2fe9dad49
enhance: Make etcd kv request timeout configurable (#28661) 
See also #28660
This pr add request timeout config item for etcd kv request timeout
 Sync the default timeout value to same value for etcdKV & tikv config

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-23 19:34:23 +08:00
XuanYang-cn
9b371067d2
feat: Add Compaction views and triggers (#27906) 
- Add Compaction l0 views
- Add Compaction scheduler
- Add Compaction triggerv2
- Add Compaction view manager

See also: #27606

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-23 17:30:25 +08:00
Bingyi Sun
4fedff6d47
feat: integrate storage v2 into the write path (#28440) 
#28378

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-23 17:26:24 +08:00
cai.zhang
31f442915b
fix: Fix bug for parsing expression that include quotes (#28416) 
issue: #28365 
Fix bug for parsing error when a string enclosed in single quotes in an
expression contains multiple double quotes.
such as:
```
expr = "tag == '\"blue\"'"
```

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-23 17:18:32 +08:00
smellthemoon
73f2bab454
enhance:add some log when create client and get component states (#28160) 
/kind improvement

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-11-22 09:12:22 +08:00
yah01
cc952e0486
enhance: optimize forwarding level0 deletions by respecting partition (#28456) 
- Cache the level 0 deletions after loading level0 segments
- Divide the level 0 deletions by partition
related: #27349

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-21 18:24:22 +08:00
aoiasd
b1fbc1f4e7
fix: datacoord l0 segment bugs (#28490) 
relate: https://github.com/milvus-io/milvus/issues/27675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-21 16:48:22 +08:00
Enwei Jiao
7445d3711c
feat: trigger compaction to handle index version (#28442) 
issue: https://github.com/milvus-io/milvus/issues/28441

---------

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-11-21 09:26:22 +08:00
wei liu
fdbca10e23
fix: Fix channel manager bg checker exit when disable auto balance (#28459) 
issue: #28454

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-20 18:20:22 +08:00
aoiasd
13a5b9f64a
fix: query l0 segment bugs (#28558) 
relate: https://github.com/milvus-io/milvus/issues/27675

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-20 17:26:23 +08:00
PowderLi
a1c505dbd5
add internal storage metrics (#28278) 
/kind improvement
issue: #28277

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-19 17:22:25 +08:00
XuanYang-cn
0be22ccacd
Fix compaction unable to notify datanode (#28409) 
See also: #28214, #28368

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-14 16:38:18 +08:00
XuanYang-cn
40d5c902b6
Enable getting multiple segments in plan result (#28350) 
Compaction plan result contained one segment for one plan. For l0
compaction would write to multiple segments, this PR expand the segments
number in plan results and refactor some names for readibility.

- Name refactory: - CompactionStateResult -> CompactionPlanResult -
CompactionResult -> CompactionSegment

See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-14 15:56:19 +08:00
XuanYang-cn
87465f07a7
Fix unstable ConsistencyHashPolicy ut (#28375) 
Fixes: #28372

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-13 19:06:20 +08:00
XuanYang-cn
a153950b10
Change channel to Interface (#27839) 
This PR changes `*channel` into RWChannel interface

See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-13 11:16:18 +08:00
wei liu
14c8a90517
Fix auto balance block channel reassign after datanode restart (#28275) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-09 19:00:25 +08:00
wei liu
b9bf910039
fix unstable auto balance config ut (#28288) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-09 10:00:22 +08:00
wei liu
7f78e1dd46
fix datacoord unstable ut (#28281) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-08 18:43:31 +08:00
cai.zhang
fd68837f84
Fix bug for setting index state when indexnode connecting failed (#28220) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-07 23:34:26 +08:00
wei liu
5b45a138b1
disable auto balance when old node exists (#28191) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-07 14:02:20 +08:00