test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 11:59:00 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
20,504 Commits 94 Branches 105 Tags 184 MiB
6b68ec55e1
Commit Graph

20504 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
XuanYang-cn
6b68ec55e1
enhance: [24]Track complex delete rates (#36958) 
See also: #36953
pr: #36957

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-10-22 10:07:27 +08:00
yanliang567
80db377586
test: [cherry pick]Add alias tests for rename collection (#37008) 
related issue: https://github.com/milvus-io/milvus/issues/36963
pr: https://github.com/milvus-io/milvus/pull/36978

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-10-21 18:45:32 +08:00
wei liu
1dcc393e54
fix: Query node panic during sending rpc to worker (#36975) (#36988) 
issue: #36976
pr: #36975

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-10-21 10:15:26 +08:00
yihao.dai
4e0f5845a1
enhance: Limit import job number (#36891) (#36892) 
issue: https://github.com/milvus-io/milvus/issues/36890

pr: https://github.com/milvus-io/milvus/pull/36891

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-10-18 18:13:25 +08:00
XuanYang-cn
7847b662d4
enhance: [cp24]Log when limit writing by deletion (#36955) 
See also: #36953
pr: #36954

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-10-18 17:47:30 +08:00
yihao.dai
8923936c9a
enhance: Support memory mode chunk cache (#35347) (#35836) 
Chunk cache supports loading raw vectors into memory.

issue: https://github.com/milvus-io/milvus/issues/35273

pr: https://github.com/milvus-io/milvus/pull/35347

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-10-18 17:03:25 +08:00
Ted Xu
2742524508
enhance: enable parallel execution of L0 compactions (#36816) (#36985) 
pr: #36816

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-10-18 16:17:25 +08:00
wei liu
b88d610e42
fix: datacoord stuck at stopping progress (#36852) (#36961) 
issue: #36868
pr: #36852
if datacoord is syncing segments to datanode, and stop datacoord
happens, datacoord's stop progress will stuck until syncing segment
finished.

This PR add ctx to syncing segment, which will failed if stopping
datacoord happens.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-10-18 14:43:24 +08:00
Ted Xu
22838a8413
enhance: Datacoord to support prioritization of compaction tasks (#36979) 
See #36550

pr: #36547 
pr: #36956

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-10-18 14:15:25 +08:00
cai.zhang
2bfd22f217
fix: [cherry-pick] Fix clustering compaction task leak (#36803) 
issue: #36686
master pr: #36800 

bug reason:
- The clustering compaction tasks on the datanode were never cleaned up.
- The clustering compaction task contains a mapping from clustering key
to buffer, this caused a large memory leak.

fix:
- clean the tasks on datanode by datacoord when clustering compaction
finished.
- reset the mapping that from clustering key to buffer on datanode when
clustering finished.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-10-17 20:43:30 +08:00
cqy123456
6934e8da3a
enhance: [2.4]use growingMmapEnabled to control the behavior of interim index, not vectorField (#36391) 
issue: https://github.com/milvus-io/milvus/issues/36392
related pr: https://github.com/milvus-io/milvus/pull/36500

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-10-17 20:23:25 +08:00
sre-ci-robot
4d2f44d536
[automated] Update Pytest image changes (#36900) 
Update Pytest image changes
See changes:
b90cda2e5d
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-10-17 18:39:24 +08:00
cqy123456
ff662308ed
fix:[2.4]Chunk Id out of range in vector BF search, after the growing index removes the vec chunks. (#36938) 
issue: https://github.com/milvus-io/milvus/issues/36871
related pr: https://github.com/milvus-io/milvus/pull/36939

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-10-17 16:51:24 +08:00
congqixia
3252d7a64c
fix: [2.4] Load original key if ts is MaxTimestamp (#36934) (#36950) 
Cherry-pick from master
pr: #36934 

Related to #36933

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-10-17 16:05:29 +08:00
yellow-shine
a9d64d6e09
Bump milvus version to v2.4.13-hotfix (#36948) 
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-10-17 14:17:28 +08:00
congqixia
dfe27ebf35
fix: [2.4] Direct forward delta exclude l0 segments (#36899) (#36914) 
Cherry-pick from master
pr: #36899

Related to #36887

Forward delete to L0 segment will return error and mark l0 segment
offline causing delegator unserviceable

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-10-17 14:11:29 +08:00
congqixia
877e9ad450
enhance: [2.4] Fill start pos & level for growing segment (#36888) (#36911) 
Cherry-pick from master
pr: #36888

Start position & level info is missing for growing segment loaded in
watch dml channel operation.

Level is important for metrics and start position is crucial for growing
exclude logic.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-10-17 14:09:32 +08:00
SimFG
8743752ac3
enhance: [2.4] force to stop buffer message when receiving the drop collection message (#36917) 
/kind improvement
pr: #36916

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-10-17 12:53:29 +08:00
wei liu
7c35c51e15
fix: Rootcoord stuck at graceful stop progress (#36881) 
issue: #34553
pr: #36880
when rootcoord trigger graceful stop progress, it will block until all
rpc finished. for create collection request, rootcoord need to block
until datacoord finish to watch all channels, but datacoord need to call
`rootcoord.Alloc` during watch channel, and rootcoord doesn't respond to
new request anymore. which cause create collection stucks, and graceful
stop progress stucks.

This PR remove the func call `rootcoord.Alloc` to solve the logic dead
lock during graceful stop progress.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-10-17 12:13:37 +08:00
yanliang567
89f28cf498
test: Add Diskann for grouping search (#36909) 
related issue: https://github.com/milvus-io/milvus/pull/36813
pr: #36853

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-10-16 16:11:24 +08:00
yanliang567
b90cda2e5d
test: [cherry pick]Add upsert in row test (#36855) 
related issue: https://github.com/milvus-io/milvus/issues/36710
pr: #36820

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-10-16 10:29:24 +08:00
nico
40259d47d1
test: update test cases (#36839) 
pr: #36853

Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-10-16 09:55:23 +08:00
yihao.dai
604e346585
enhance: Enhance segment log (#36848) (#36849) 
/kind improvement

pr: https://github.com/milvus-io/milvus/pull/36848

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-10-15 20:43:30 +08:00
yihao.dai
f8d9c24256
enhance: Ignore db not found error in quota center (#36821) (#36850) 
In quota center, ignore the "DB not found error" to prevent it from
affecting the rate limiting of other databases.

/kind improvement

pr: https://github.com/milvus-io/milvus/pull/36821

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-10-15 20:41:24 +08:00
yellow-shine
85ce66843b
enhance: [ci] separate ccache by branch for e2e pipeline (#36873) 
https://github.com/milvus-io/milvus/pull/36870

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-10-15 15:25:28 +08:00
yellow-shine
f67f69a9c1
enhance: able to upload pytest logs when test is done (#36861) 
https://github.com/milvus-io/milvus/pull/36842

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-10-15 11:49:23 +08:00
SimFG
6b9e28bc8f
enhance: [2.4] update the expr version to support automatic conversion of variable types (#36847) 
/kind improvement
- pr: #36832

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-10-15 10:55:23 +08:00
Xiaofan
44564f0403
fix: make sure alias is cached (#36808) 
see also #36806

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-10-13 17:39:22 +08:00
yanliang567
a645ab87a0
test: [cherry pick]Add json key in [] test to verify issue (#36810) 
related issue: https://github.com/milvus-io/milvus/issues/36718
pr: https://github.com/milvus-io/milvus/pull/36738

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-10-12 18:03:21 +08:00
sre-ci-robot
523a3a0526
[automated] Bump milvus version to v2.4.13 (#36802) 
Bump milvus version to v2.4.13
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-10-12 16:49:31 +08:00
yellow-shine
272e3127e7
enhance: change way to bump milvus bersion (#36779) 
https://github.com/milvus-io/milvus/pull/36776

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-10-12 10:41:25 +08:00
SimFG
1d9c7462ba
enhance: [2.4] support to execute the method which contains the ctx param (#36798) 
/kind improvement
- pr: #36797

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-10-11 23:17:21 +08:00
SimFG
0fe549adf1
enhance: [2.4] gc in the snapshot kv (#36793) 
- issue: #36770
- pr: #36792

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-10-11 22:13:21 +08:00
jaime
43b48a0f9b
enhance: skip alter operation when no change are detected (#36786) 
issue: #36784 
pr: #36785

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-10-11 21:43:21 +08:00
jaime
4110f35f1a
fix: may exceed max tnx in etcd operations (#36773) 
issue: #36772
pr: #36775

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-10-11 19:51:22 +08:00
congqixia
a57626afa0
fix: [2.4] Use metrics values instead of hardcode mock values (#36783) 
Cherry-pick from master
pr: #36782
Resolves #36781

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-10-11 19:49:21 +08:00
cqy123456
58b198c785
enhance: [2.4]support sparse vector mmap in growing segment type (#36565) 
issue: https://github.com/milvus-io/milvus/issues/32984
related pr: https://github.com/milvus-io/milvus/pull/36566

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-10-11 17:23:21 +08:00
XuanYang-cn
e976b41f97
fix: Remove enableLevelZeroSegment config (#36507) 
See also: #36504
pr: #36535

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-10-11 16:41:21 +08:00
XuanYang-cn
28de6b86ba
fix: [24]fail to init fg clears flushTs so that slows flush (#36741) 
See also: #36709
pr: #36740

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-10-11 16:39:21 +08:00
yihao.dai
7ee46c582e
enhance: Support new RESTful URL to describe import job (#36753) (#36754) 
Support the new RESTful URL for retrieving/describing import progress:
`/v2/vectordb/jobs/import/describe`.
Deprecate the old URL: `/v2/vectordb/jobs/import/get_progress`.

issue: https://github.com/milvus-io/milvus/issues/36752

pr: https://github.com/milvus-io/milvus/pull/36753

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-10-11 16:03:22 +08:00
congqixia
bacbfae542
enhance: Bump milvus & proto version to v2.4.13 (#36758) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-10-11 16:01:22 +08:00
yihao.dai
a4ef93457d
enhance: Optimize import scheduling and add time cost metric (#36601) (#36684) 
1. Optimize import scheduling strategic:
a. Revise slot weights, calculating them based on the number of files
and segments for both import and pre-import tasks.
b. Ensure that the DN executes tasks in ascending order of task ID.
2. Add time cost metric and log.

issue: https://github.com/milvus-io/milvus/issues/36600,
https://github.com/milvus-io/milvus/issues/36518

pr: https://github.com/milvus-io/milvus/pull/36601

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-10-11 10:27:22 +08:00
congqixia
b24788b2c7
enhance: [2.4] Add balance report log for qc balancer (#36749) 
Cherry pick from master
pr: #36747 
Related to #36746

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-10-11 10:25:24 +08:00
SimFG
548f8e80c3
enhance: [2.4] the estimate method when loading the collection (#36728) 
- pr: #36307
- issue: #36530

Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
 2024-10-11 10:20:45 +08:00
XuanYang-cn
c69af44472
enhance: [24]Add more info in logs (#36732) 
pr: #36535

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-10-10 17:51:25 +08:00
yellow-shine
e8c40e94ad
enhance: use new implementation of e2e pipeline (#36687) 
https://github.com/milvus-io/milvus/issues/36348
https://github.com/milvus-io/milvus/pull/36347

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-10-10 17:11:21 +08:00
aoiasd
eaa948752b
enhance: [Cherry-Pick] UpdateSegmentsInfo should update remaining segment info even if some one not exist (#36729) 
pr: https://github.com/milvus-io/milvus/pull/36726

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-10-10 15:11:20 +08:00
zhagnlu
02bd916fa4
fix: fix: fix json in [] expr bug (#36722) 
pr: #36721

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-10-09 21:53:20 +08:00
yihao.dai
9cb5396cf6
enhance: Use common gc config (#36668) (#36670) 
Use the GC config from `common` and remove the GC config from
`queryNode`.

issue: https://github.com/milvus-io/milvus/issues/36667

pr: https://github.com/milvus-io/milvus/pull/36668

related pr: https://github.com/milvus-io/milvus/pull/34949

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-10-09 19:49:20 +08:00
congqixia
3a80d1f602
enhance: [2.4] Add streaming forward policy switch for delegator (#36330) (#36712) 
Cherry pick from master
pr: #36330
Related to #35303

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-10-09 17:41:20 +08:00