sre-ci-robot
af2da8d8d6
[automated] Update Pytest image changes ( #36845 )
...
Update Pytest image changes
See changes:
c96bbe19basre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-14 19:11:23 +08:00
Zhen Ye
f46c3acea9
fix: heap buffer overflow when unittest at index wrapper ( #36838 )
...
issue: #35852
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-14 18:13:22 +08:00
aoiasd
5ec4163d0f
feat: support bm25 logs mixcompaction ( #36072 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-14 16:57:22 +08:00
yanliang567
c96bbe19ba
test: Add upsert in rows tests ( #36820 )
...
related issue: #36710
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-14 16:41:22 +08:00
Zhen Ye
f0f5147aef
fix: streaming consumer may get stucked when handler is un-consumed ( #36818 )
...
issue: #36378
Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-14 15:23:23 +08:00
Zhen Ye
8905b042f1
fix: add proportion for capacity seal policy in streaming flusher ( #36761 )
...
issue: #36760
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-14 14:47:22 +08:00
yellow-shine
00e7e8c661
enhance: enable macos checker ( #36833 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-10-14 14:09:21 +08:00
Bingyi Sun
3a09b438c2
fix: fix macos code checker ( #36817 )
...
https://github.com/milvus-io/milvus/issues/36829
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-10-14 11:11:51 +08:00
yihao.dai
d230b91bd1
enhance: Add PreallocatedSegmentIDs for the compaction task ( #36734 )
...
Add `PreallocatedSegmentIDs` field to the compaction task, allowing the
`ResultSegments` in the compaction task to represent the final segments
produced by the compaction.
issue: https://github.com/milvus-io/milvus/issues/36733
also related: https://github.com/milvus-io/milvus/issues/36686
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-13 17:59:21 +08:00
Buqian Zheng
383350c120
feat: added more checks for function creation check ( #36766 )
...
issue: https://github.com/milvus-io/milvus/issues/35853
* BM25 Function now takes no params, k1, b should be passed via index
params
* support BM25 full text search when metric type is not present in
search request
* add more strict validation with functions at collection creation time
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-13 17:43:22 +08:00
Buqian Zheng
16b533cbf0
feat: Restful support for BM25 function ( #36713 )
...
issue: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-13 17:41:21 +08:00
sre-ci-robot
e170991a10
[automated] Update Knowhere Commit ( #36823 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-13 01:21:20 +08:00
Min Tian
ef0c649bda
enhance: update knowhere version to support diskann iterator ( #36813 )
...
issue: #36812
Signed-off-by: min.tian <min.tian.cn@gmail.com>
2024-10-12 18:05:22 +08:00
Buqian Zheng
82c5cf2fa2
feat: add bulk insert support for Functions ( #36715 )
...
issue: https://github.com/milvus-io/milvus/issues/35853 and
https://github.com/milvus-io/milvus/issues/35856
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-12 17:19:20 +08:00
sre-ci-robot
d45e2677b9
[automated] Bump milvus version to v2.4.13 ( #36801 )
...
Bump milvus version to v2.4.13
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-12 15:07:24 +08:00
Bingyi Sun
a75bb85f3a
feat: support chunked column for sealed segment ( #35764 )
...
This PR splits sealed segment to chunked data to avoid unnecessary
memory copy and save memory usage when loading segments so that loading
can be accelerated.
To support rollback to previous version, we add an option
`multipleChunkedEnable` which is false by default.
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-10-12 15:04:52 +08:00
jaime
5713620825
enhance: skip alter operation when no change are detected ( #36785 )
...
issue: #36784
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-12 11:25:20 +08:00
yellow-shine
5b2658327d
enhance: change way to bump milvus bersion ( #36776 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-10-12 10:41:25 +08:00
SimFG
03a78ecc3d
enhance: gc in the snapshot kv ( #36792 )
...
issue: #36770
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-10-12 04:03:20 +08:00
SimFG
7acf724185
enhance: support to execute the method which contains the ctx param ( #36797 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-10-12 00:01:20 +08:00
congqixia
bfe5c651d6
doc: Update go version in DEVELOPMENT.md ( #36796 )
...
go 1.22 is needed for master branch. Update document for developer guide
markdown.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-11 21:25:20 +08:00
smellthemoon
5e1c392243
enhance: support null and default value in restful api ( #35825 )
...
#31728
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-11 21:23:20 +08:00
congqixia
116d7c18cc
fix: Use metrics values instead of hardcode mock values ( #36782 )
...
Resolves #36781
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-11 20:09:20 +08:00
XuanYang-cn
794e3ab7e5
fix: fail to init fg clears flushTs so that slows flush ( #36740 )
...
See also: #36709
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-11 17:37:04 +08:00
yihao.dai
0751c508de
enhance: Support new RESTful URL to describe import job ( #36753 )
...
Support the new RESTful URL for retrieving/describing import progress:
`/v2/vectordb/jobs/import/describe`.
Deprecate the old URL: `/v2/vectordb/jobs/import/get_progress`.
issue: https://github.com/milvus-io/milvus/issues/36752
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-11 16:07:21 +08:00
CharlesFeng
7c8b71e26c
fix: BinlogDeserializeReader leak in mix_compactor.go ( #36270 )
...
https://github.com/milvus-io/milvus/issues/36269
Signed-off-by: fengjun2016 <jornfeng@gmail.com>
2024-10-11 15:41:20 +08:00
yanliang567
621dbc9107
test: Add json key in [] test to verify issue 36718 ( #36738 )
...
related issue: #36718
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-11 14:29:21 +08:00
cai.zhang
d1060c0e05
enhance: Update antlr version and refine parsing not in ( #36745 )
...
issue: #36672
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-11 14:03:21 +08:00
congqixia
3fe0f82923
enhance: Add balance report log for qc balancer ( #36747 )
...
Related to #36746
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-11 10:25:24 +08:00
aoiasd
db34572c56
feat: support load and query with bm25 metric ( #36071 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-11 10:23:20 +08:00
sre-ci-robot
90285830de
[automated] Update Pytest image changes ( #36755 )
...
Update Pytest image changes
See changes:
f7b811450dsre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-11 09:29:19 +08:00
zhagnlu
b1e678dcba
fix: fix json in [] expr bug ( #36721 )
...
#36718
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-10-11 01:11:20 +08:00
Buqian Zheng
f7b811450d
feat: add enable_tokenizer params to VarChar field ( #36480 )
...
issue: #35922
add an enable_tokenizer param to varchar field: must be set to true so
that a varchar field can enable_match or used as input of BM25 function
---------
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-10 20:33:21 +08:00
XuanYang-cn
290ceb4e84
enhance: Add more info in logs ( #36731 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-10 17:51:25 +08:00
aoiasd
a5f98e4cb3
enhance: UpdateSegmentsInfo should update remaining segment info even if some one not exist ( #36726 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-10 15:17:19 +08:00
Ted Xu
62679efa0c
enhance: remove unnecessary clone in meta cache ( #36628 )
...
See #36627
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-10-10 15:05:20 +08:00
yihao.dai
3685edb264
enhance: Use common gc config ( #36668 )
...
Use the GC config from `common` and remove the GC config from
`queryNode`.
issue: https://github.com/milvus-io/milvus/issues/36667
related pr: https://github.com/milvus-io/milvus/pull/34949
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-09 19:47:19 +08:00
Ted Xu
5fc731795b
enhance: Datacoord to support prioritization of compaction tasks ( #36547 )
...
See #36550
This PR made 2 changes:
1. Introducing a prioritization mechanism, if
`dataCoord.compaction.taskPrioritizer` is set to `level`, compaction
tasks are always executed as the priority of L0>Mix>Clustering
2. `dataCoord.compaction.maxParallelTaskNum` now controls the
parallelism of executing tasks, not the task number of queue +
executing.
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-10-09 19:11:20 +08:00
nico
e1b511a219
enhance: open mmap on standalone-kafka in ci-e2e ( #36688 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-10-09 19:03:19 +08:00
Zhen Ye
efb10d3b86
fix: streaming metrics do not clear ( #36701 )
...
issue: #33285
Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-09 17:51:20 +08:00
SimFG
130a923dec
enhance: the estimate method when loading the collection ( #36307 )
...
- issue: #36530
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
Co-authored-by: xianliang.li <xianliang.li@zilliz.com>
2024-10-09 17:35:19 +08:00
XuanYang-cn
c84bdfa766
fix: raise l0 compaction memory ratio to 0.5 ( #36690 )
...
5 percent of free memory is too less for l0 compaction. This pr will
raise it to 50 percent.
See also: #36614
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-09 17:19:24 +08:00
congqixia
c3d910756b
enhance: Update knowhere commit to fix mac compilation ( #36706 )
...
Related to zilliztech/knowhere#879
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-09 16:05:20 +08:00
yihao.dai
0fc2a4aa53
enhance: Optimize import scheduling and add time cost metric ( #36601 )
...
1. Optimize import scheduling strategic:
a. Revise slot weights, calculating them based on the number of files
and segments for both import and pre-import tasks.
b. Ensure that the DN executes tasks in ascending order of task ID.
2. Add time cost metric and log.
issue: https://github.com/milvus-io/milvus/issues/36600 ,
https://github.com/milvus-io/milvus/issues/36518
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-09 14:41:20 +08:00
aoiasd
2c3a8b7dea
enhance: fix skip msg log with empty position info ( #36626 )
...
All skiped message position info was nil, should print BeginTs or
MessageID instead.
relate: https://github.com/milvus-io/milvus/issues/36569
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-09 12:01:20 +08:00
Zhen Ye
addced27f9
fix: rocksmq consume too slow when the channel is full ( #36617 )
...
issue: #36569
Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-09 11:59:31 +08:00
yihao.dai
4f50b7dbab
enhance: Swap the position of import job states ( #36699 )
...
issue: https://github.com/milvus-io/milvus/issues/36698
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-09 11:29:19 +08:00
Patrick Weizhi Xu
90a0e90732
enhance: remove warnings when there is no partition key set ( #36674 )
...
issue: #36673
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-10-09 10:27:20 +08:00
cai.zhang
fc8b5ab791
enhance: Convert unincode to ascii to improving expression parsing efficiency ( #36675 )
...
issue: #36672
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-09 09:23:24 +08:00
Zhen Ye
2ec6e602d6
enhance: add streaming client metrics ( #36523 )
...
issue: #33285
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-08 21:25:19 +08:00
