nico
40259d47d1
test: update test cases ( #36839 )
...
pr: #36853
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-10-16 09:55:23 +08:00
yihao.dai
604e346585
enhance: Enhance segment log ( #36848 ) ( #36849 )
...
/kind improvement
pr: https://github.com/milvus-io/milvus/pull/36848
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-15 20:43:30 +08:00
yihao.dai
f8d9c24256
enhance: Ignore db not found error in quota center ( #36821 ) ( #36850 )
...
In quota center, ignore the "DB not found error" to prevent it from
affecting the rate limiting of other databases.
/kind improvement
pr: https://github.com/milvus-io/milvus/pull/36821
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-15 20:41:24 +08:00
yellow-shine
85ce66843b
enhance: [ci] separate ccache by branch for e2e pipeline ( #36873 )
...
https://github.com/milvus-io/milvus/pull/36870
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-10-15 15:25:28 +08:00
yellow-shine
f67f69a9c1
enhance: able to upload pytest logs when test is done ( #36861 )
...
https://github.com/milvus-io/milvus/pull/36842
---------
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-10-15 11:49:23 +08:00
SimFG
6b9e28bc8f
enhance: [2.4] update the expr version to support automatic conversion of variable types ( #36847 )
...
/kind improvement
- pr: #36832
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-10-15 10:55:23 +08:00
Xiaofan
44564f0403
fix: make sure alias is cached ( #36808 )
...
see also #36806
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-10-13 17:39:22 +08:00
yanliang567
a645ab87a0
test: [cherry pick]Add json key in [] test to verify issue ( #36810 )
...
related issue: https://github.com/milvus-io/milvus/issues/36718
pr: https://github.com/milvus-io/milvus/pull/36738
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-12 18:03:21 +08:00
sre-ci-robot
523a3a0526
[automated] Bump milvus version to v2.4.13 ( #36802 )
...
Bump milvus version to v2.4.13
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-12 16:49:31 +08:00
yellow-shine
272e3127e7
enhance: change way to bump milvus bersion ( #36779 )
...
https://github.com/milvus-io/milvus/pull/36776
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-10-12 10:41:25 +08:00
SimFG
1d9c7462ba
enhance: [2.4] support to execute the method which contains the ctx param ( #36798 )
...
/kind improvement
- pr: #36797
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-10-11 23:17:21 +08:00
SimFG
0fe549adf1
enhance: [2.4] gc in the snapshot kv ( #36793 )
...
- issue: #36770
- pr: #36792
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-10-11 22:13:21 +08:00
jaime
43b48a0f9b
enhance: skip alter operation when no change are detected ( #36786 )
...
issue: #36784
pr: #36785
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-11 21:43:21 +08:00
jaime
4110f35f1a
fix: may exceed max tnx in etcd operations ( #36773 )
...
issue: #36772
pr: #36775
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-11 19:51:22 +08:00
congqixia
a57626afa0
fix: [2.4] Use metrics values instead of hardcode mock values ( #36783 )
...
Cherry-pick from master
pr: #36782
Resolves #36781
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-11 19:49:21 +08:00
cqy123456
58b198c785
enhance: [2.4]support sparse vector mmap in growing segment type ( #36565 )
...
issue: https://github.com/milvus-io/milvus/issues/32984
related pr: https://github.com/milvus-io/milvus/pull/36566
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-10-11 17:23:21 +08:00
XuanYang-cn
e976b41f97
fix: Remove enableLevelZeroSegment config ( #36507 )
...
See also: #36504
pr: #36535
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-11 16:41:21 +08:00
XuanYang-cn
28de6b86ba
fix: [24]fail to init fg clears flushTs so that slows flush ( #36741 )
...
See also: #36709
pr: #36740
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-11 16:39:21 +08:00
yihao.dai
7ee46c582e
enhance: Support new RESTful URL to describe import job ( #36753 ) ( #36754 )
...
Support the new RESTful URL for retrieving/describing import progress:
`/v2/vectordb/jobs/import/describe`.
Deprecate the old URL: `/v2/vectordb/jobs/import/get_progress`.
issue: https://github.com/milvus-io/milvus/issues/36752
pr: https://github.com/milvus-io/milvus/pull/36753
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-11 16:03:22 +08:00
congqixia
bacbfae542
enhance: Bump milvus & proto version to v2.4.13 ( #36758 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-11 16:01:22 +08:00
yihao.dai
a4ef93457d
enhance: Optimize import scheduling and add time cost metric ( #36601 ) ( #36684 )
...
1. Optimize import scheduling strategic:
a. Revise slot weights, calculating them based on the number of files
and segments for both import and pre-import tasks.
b. Ensure that the DN executes tasks in ascending order of task ID.
2. Add time cost metric and log.
issue: https://github.com/milvus-io/milvus/issues/36600 ,
https://github.com/milvus-io/milvus/issues/36518
pr: https://github.com/milvus-io/milvus/pull/36601
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-11 10:27:22 +08:00
congqixia
b24788b2c7
enhance: [2.4] Add balance report log for qc balancer ( #36749 )
...
Cherry pick from master
pr: #36747
Related to #36746
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-11 10:25:24 +08:00
SimFG
548f8e80c3
enhance: [2.4] the estimate method when loading the collection ( #36728 )
...
- pr: #36307
- issue: #36530
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-10-11 10:20:45 +08:00
XuanYang-cn
c69af44472
enhance: [24]Add more info in logs ( #36732 )
...
pr: #36535
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-10 17:51:25 +08:00
yellow-shine
e8c40e94ad
enhance: use new implementation of e2e pipeline ( #36687 )
...
https://github.com/milvus-io/milvus/issues/36348
https://github.com/milvus-io/milvus/pull/36347
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-10-10 17:11:21 +08:00
aoiasd
eaa948752b
enhance: [Cherry-Pick] UpdateSegmentsInfo should update remaining segment info even if some one not exist ( #36729 )
...
pr: https://github.com/milvus-io/milvus/pull/36726
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-10 15:11:20 +08:00
zhagnlu
02bd916fa4
fix: fix: fix json in [] expr bug ( #36722 )
...
pr: #36721
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-10-09 21:53:20 +08:00
yihao.dai
9cb5396cf6
enhance: Use common gc config ( #36668 ) ( #36670 )
...
Use the GC config from `common` and remove the GC config from
`queryNode`.
issue: https://github.com/milvus-io/milvus/issues/36667
pr: https://github.com/milvus-io/milvus/pull/36668
related pr: https://github.com/milvus-io/milvus/pull/34949
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-09 19:49:20 +08:00
congqixia
3a80d1f602
enhance: [2.4] Add streaming forward policy switch for delegator ( #36330 ) ( #36712 )
...
Cherry pick from master
pr: #36330
Related to #35303
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-09 17:41:20 +08:00
XuanYang-cn
05f96f5298
fix: [24]raise l0 compaction memory ratio to 0.5 ( #36691 )
...
5 percent of free memory is too less for l0 compaction. This pr will
raise it to 50 percent.
See also: #36614
pr: #36690
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-09 17:19:24 +08:00
yihao.dai
93a4574a35
Add buildIndex state for import job ( #36705 )
...
issue: https://github.com/milvus-io/milvus/issues/36698
pr: https://github.com/milvus-io/milvus/pull/35868 ,
https://github.com/milvus-io/milvus/pull/36699
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-09 16:57:20 +08:00
XuanYang-cn
0ff8e13232
fix: [24]Remove neighbors if compactTo is unindexed ( #36503 ) ( #36694 )
...
See also: #36360
pr: #36503
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-09 15:43:20 +08:00
Zhen Ye
bcc661dbd0
fix: rocksmq consume too slow if the channel is full ( #36618 )
...
issue: #36569
pr: #36617
Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-09 11:59:31 +08:00
Patrick Weizhi Xu
6e93601b2c
enhance: [2.4] remove warnings when there is no partition key set ( #36679 )
...
issue: #36673
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit db60f7cf2a56656b101067a2dbebd4dd32ed6ed2)
2024-10-09 09:35:20 +08:00
cai.zhang
f7302a78fb
enhance: [cherry-pick]Convert unincode to ascii to improving expression parsing efficiency ( #36676 )
...
issue: #36672
master pr: #36675
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-09 09:23:24 +08:00
sre-ci-robot
a081bf394d
[automated] Update Pytest image changes ( #36689 )
...
Update Pytest image changes
See changes:
f10806650asre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-09 08:51:19 +08:00
congqixia
1955738ab8
enhance: [2.4] Produce messages of multiple topics in parallel ( #36344 ) ( #36462 )
...
Cherry-pick from master
pr: #36344
Related to #36343
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-08 18:31:20 +08:00
congqixia
9073f6281e
fix: [2.4] Add defer Unpin when error happens ( #36620 ) ( #36665 )
...
Cherry-pick from master
pr: #36620
Resolves : #36619
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-08 18:29:20 +08:00
Buqian Zheng
03fba95284
fix: [2.4] improve mmap related code in Column.h ( #36521 )
...
pr: #36183
issue: #36182
---------
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-08 18:27:19 +08:00
nico
f10806650a
test: update pymilvus version and test case ( #36590 )
...
pr: #36589
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-10-08 17:25:24 +08:00
jaime
da2d3fb430
enhance: enable manual compaction for collections without indexes ( #36581 )
...
issue: #36576
pr: #36577
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-08 14:51:20 +08:00
jaime
bf671786d9
enhance: decrease CPU overhead when calculating index file size ( #36580 )
...
issue: #36578
pr: #36579
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-08 14:29:25 +08:00
yihao.dai
86004d852e
enhance: Reopen chunk cache cpp ut ( #33622 ) ( #33623 )
...
issue: https://github.com/milvus-io/milvus/issues/33210
pr: https://github.com/milvus-io/milvus/pull/33622
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-08 11:21:22 +08:00
yihao.dai
29c1d5042a
fix: Fix import segments leak in segment manager ( #36602 ) ( #36631 )
...
Directly add import segments from the meta, eliminating the dependency
on the segment manager.
issue: https://github.com/milvus-io/milvus/issues/34648
pr: https://github.com/milvus-io/milvus/pull/36602
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-08 10:15:22 +08:00
wei liu
2428adea3b
enhance: Enable balance on querynode with different mem capacity ( #36466 ) ( #36625 )
...
issue: #36464
pr: #36466
This PR enable balance on querynode with different mem capacity, for
query node which has more mem capactity will be assigned more records,
and query node with the largest difference between assignedScore and
currentScore will have a higher priority to carry the new segment.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-30 18:11:18 +08:00
congqixia
385bfc2639
enhance: [2.4] Unify InboundLabel case using metrics.Label ( #36613 ) ( #36616 )
...
Cherry pick from master
pr: #36613
Previous label case broken by #36107 , this PR make all inbound label
using label constants from metrics package.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-30 16:19:17 +08:00
wei liu
f8dfd0de53
fix: Skip unnecessary query node health check in proxy ( #36491 ) ( #36553 )
...
issue: #36490
pr: #36491
After the query node changes from a delegator to a worker, proxy should
skip this querynode's health check.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-30 14:27:16 +08:00
wei liu
4120320074
enhance: make TransferChannel/TransferSegment idempotent ( #36489 ) ( #36552 )
...
issue: #36488
pr: #36489
when call TransferChannel/TransferSegment, querycoord will generate and
submit balance task to scheduler, if segment/channel's task already
exist in scheduler, submit task will failed.
to make TransferChannel/TransferSegment idempotent, we should skip to
submit if task already exist in scheduler.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-30 14:25:26 +08:00
yihao.dai
c5088b8527
enhance: Add metrics to monitor import throughput and imported rows ( #36519 ) ( #36588 )
...
issue: https://github.com/milvus-io/milvus/issues/36518
pr: https://github.com/milvus-io/milvus/pull/36519
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-30 10:03:18 +08:00
SimFG
58a763c529
enhance: [2.4] avoid to create many timer object in the target ( #36573 )
...
/kind improvement
- pr: #36570
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-09-29 19:27:16 +08:00
