test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 03:48:37 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
19,325 Commits 94 Branches 105 Tags 184 MiB
bdc70dfc6a
Commit Graph

19325 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
sre-ci-robot
2d9de233fc
[automated] Update Knowhere Commit (#31089) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-03-07 12:05:02 +08:00
cai.zhang
951c91c9b4
fix: [skip e2e]Increase graceful time to ensure that bounded search executing normally (#31064) 
issue: #31063 
Increase the graceful time to ensure that bounded query requests can be
executed normally in the event of RootCoord failure.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-03-07 10:53:00 +08:00
yihao.dai
8ff8fefdc1
fix: Fix errors in the Index service APIs are ignored (#31077) 
In Index service APIs, return error if occurs instead of always
returning nil. Additionally, add more tests to cover this scenario.

issue: https://github.com/milvus-io/milvus/issues/31069,
https://github.com/milvus-io/milvus/issues/31027

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-06 22:11:00 +08:00
congqixia
196f0c1e1d
fix: Skip invalid compaction plan (#31045) 
See also #31044

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-06 21:36:59 +08:00
congqixia
007fab183c
enhance: Skip partition key name check if feature not enabled (#31067) 
Skip partition key naming & hash value pre process if collection schema
does not have partition key
The PR removes mislead warning when collection has no partition key

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-06 19:33:00 +08:00
MrPresent-Han
d0eeea4b44
fix: reduce incorrectly for group-by with offset(#30828) (#30882) 
related: #30828

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-03-06 16:47:00 +08:00
Bingyi Sun
fd17a5f050
fix: check collection lazy load prop using schema (#30992) 
issue: https://github.com/milvus-io/milvus/issues/30361

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-03-06 16:19:01 +08:00
Bingyi Sun
df7aafa3ec
fix: filter mmap key when checking index params (#31030) 
issue: https://github.com/milvus-io/milvus/issues/31031

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-03-06 16:03:00 +08:00
Jiquan Long
a88c896733
enhance: purge client infos periodically (#31037) 
https://github.com/milvus-io/milvus/issues/31007

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-03-06 12:50:59 +08:00
sre-ci-robot
3c9ffdedf3 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-03-05 12:00:52 +00:00
XuanYang-cn
def72947c7
fix: Trigger l0 compaction when l0 views don't change (#30729) 
Trigger l0 compaction when l0 views don't change

So that leftover l0 segments would be compacted in the end.

1. Refresh LevelZero plans in comactionPlanHandler, remove the meta
dependency
of compaction trigger v2
2. Add ForceTrigger method for CompactionView interface
3. rename mu to taskGuard
4. Add a new TriggerTypeLevelZeroViewIDLE
5. Add an idleTicker for compaction view manager

See also: #30098, #30556

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-03-05 16:37:00 +08:00
congqixia
30398d4b71
enhance: Fix misleading log content & possible nil panic (#31021) 
- Change load field log from "dy pool" to "load pool"
- Also defer delete when there is no error

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-05 16:33:00 +08:00
sre-ci-robot
c047f09110
[automated] Update Knowhere Commit (#31015) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-03-05 16:31:00 +08:00
Bingyi Sun
e3cce11dd9
fix: data race in querynode task test (#31019) 
issue: https://github.com/milvus-io/milvus/issues/31022

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-03-05 16:26:59 +08:00
yihao.dai
3b66c17279
feat: Revise the RESTful bulk insert API (#29698) 
Revise the RESTful bulk insert API from version 1 to version 2.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-05 15:03:00 +08:00
congqixia
8c2615f840
enhance: Add unit(seconds) for new added connection manager param (#31023) 
See also #31007 #31008 #31009

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-05 14:50:59 +08:00
zhagnlu
b9775a1816
fix: add GetSegments optimization to avoid meta mutex competition (#31025) 
#30835

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-03-05 14:47:00 +08:00
Xiaofan
4bda6c33ad
fix: binary vector should not limit dimension to 32768 (#30676) 
all the vector dimension check should happen on collection creation but
not index build
fix #30285

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-03-05 14:21:00 +08:00
congqixia
1936aa4caa
enhance: Check channel cp lag before generate compaction task (#30997) 
See also #30996

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-05 13:39:01 +08:00
aoiasd
0dacf0172e
enhance: access log support print output fields of Query or Search (#30768) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-03-05 11:54:58 +08:00
congqixia
3b5ce73ded
enhance: Change proxy connection manager to concurrent safe (#31008) 
See also #31007

This PR:
- Add param item for connection manager behavior: TTL & check interval
- Change clientInfo map to concurrent map

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-05 10:39:00 +08:00
chyezh
8f7019468f
fix: starve lock caused by slow GetCompactionTo method when too much segments (#30963) 
issue: #30823

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-03-05 10:04:59 +08:00
congqixia
52540fecb2
enhance: Add back load memory factor when esitmating memory resource (#30994) 
Segment load memory usage is underestimated due to removing the load
memroy factor. This PR adds it back to protect querynode OOM during some
extreme memory cases.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-04 19:54:59 +08:00
congqixia
9b3005f1be
enhance: Avoid create schema helper for each read task (#30981) 
See also #30806

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-04 19:39:00 +08:00
elstic
bd063a352d
test: add test cases for mmap (#30942) 
issue: https://github.com/milvus-io/milvus/issues/30940

Signed-off-by: elstic <hao.wang@zilliz.com>
 2024-03-04 19:33:01 +08:00
Ted Xu
71adafa933
enhance: adding a streaming deserialize reader for binlogs (#30860) 
See #30863

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-03-04 19:31:09 +08:00
jaime
4b0c3dd377
enhance: index meta use independent rather than global meta lock (#30869) 
issue: https://github.com/milvus-io/milvus/issues/30837

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-03-04 16:56:59 +08:00
sre-ci-robot
3dc5e38240
[automated] Update Knowhere Commit (#30989) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-03-04 16:34:59 +08:00
wei liu
fa73520b57
fix: Make datacoord client retry on index api (#30654) 
issue: #20553

This PR add retry on all interface which belong to indexcoord in milvus
2.2 and. move to data coord in milvus 2.3, to prevent meet
`unimplemented` error during rolling upgrade from milvus 2.2 to 2.3.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-03-04 15:32:59 +08:00
sre-ci-robot
882b50c5c1
[automated] Update Pytest image changes (#30852) 
Update Pytest image changes
See changes:
7f78e9d40d
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-03-04 09:55:00 +08:00
cai.zhang
f6ff2588cd
enhance: Optimize DescribeIndex to reduce lock contention (#30939) 
issue: #29313 
issue: #30443

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-03-03 19:00:59 +08:00
PowderLi
e39f46aa38
fix: use GetBinaryField incorrectly (#30969) 
issue: #30980 #22837
2.3 pr: #30873

should use GetBFloat16Field, GetBfloat16Field rather than GetBinaryField

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-03-03 17:58:59 +08:00
zhuwenxing
d98a5e441f
test: add restful v2 test (#30984) 
add restful v2 test

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-03-02 11:03:03 +08:00
nico
d8164c43d2
test: update test cases (#30777) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-03-02 11:01:13 +08:00
yihao.dai
0a2c255630
enhance: Reduce the memory usage of the timeTickSender (#30968) 
In the cache of the timeTickSender, retain only the latest stats instead
of storing stats for every time tick.

issue: https://github.com/milvus-io/milvus/issues/30967

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-02 10:13:01 +08:00
MrPresent-Han
29f44f840a
enhance: refine groupBy error msg(#29968) (#30920) 
related: #29968

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-03-01 18:53:03 +08:00
yihao.dai
a434d33e75
feat: Add import scheduler and manager (#29367) 
This PR introduces novel managerial roles for importv2:
1. ImportMeta: To manage all the import tasks;
2. ImportScheduler: To process tasks and modify their states;
3. ImportChecker: To ascertain the completion of all tasks and instigate
relevant operations.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-01 18:31:02 +08:00
groot
85de56e894
fix: Clean kafka default configuration (#30924) 
issue: #30917

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2024-03-01 18:17:03 +08:00
Bingyi Sun
7783098ddd
feat: support lazy load on querycoord (#30372) 
https://github.com/milvus-io/milvus/issues/30361

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-03-01 18:15:29 +08:00
wayblink
f3c56c83ab
fix: Fix binlog_io metric name conflict (#30689) 
follow: #29725

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-03-01 18:13:02 +08:00
zhagnlu
8b25ecffff
fix: add pk sort for same distance of hybridsearch (#30901) 
#29923

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-03-01 18:09:02 +08:00
cai.zhang
41f1e6dcad
enhance:Add integration test for searching when coord down (#30717) 
issue: #29772

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-03-01 17:07:00 +08:00
cai.zhang
1aa97a5c21
enhance: Support more relational operators for binary expressions (#30902) 
issue: #30677

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-03-01 16:57:00 +08:00
SimFG
ee8d6f236c
enhance: make the watch dm channel request better compatibility (#30952) 
issue: #30938

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-03-01 16:07:37 +08:00
chyezh
dd957cf9e3
enhance: add configurable memory index load predict memory usage factor (#30561) 
related pr: https://github.com/milvus-io/milvus/pull/30475

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-03-01 15:23:00 +08:00
wei liu
cc46d6bafc
fix: segment version doesn't update as expected (#30951) 
issue: #30950

due to segment version doesn't update as expected.
This PR will update segment version until segment become loaded

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-03-01 14:21:10 +08:00
congqixia
6387403639
fix: Prevent clone when selecting segments from meta (#30928) 
See also #30538

Previously the `SelectSegments` changed to clone all return value
preventing possible update to returned info.

Since meta is implemented following COW rules, this shall not happen and
any update on segment shall have copy before it.

This PR:
- Remove clone for read-only Get segment info
- Add Segment Operator abstraction for changing segment
- Implemnt COW for updating MaxRowNum

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-01 13:59:02 +08:00
congqixia
385dec3b69
fix: index attr caches wrong result variable (#30960) 
See also #30757 #30756

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-01 13:17:00 +08:00
aoiasd
2180e2cfae
fix: wrong segment binlog path cause load segment failed #30726 (#30959) 
DataCoord GetSegmentInfo should return binlog info with logpath instead
logid when segment merge child segment's
binlog.
relate: https://github.com/milvus-io/milvus/issues/30366
https://github.com/milvus-io/milvus/issues/30165
https://github.com/milvus-io/milvus/issues/30550

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-03-01 13:09:01 +08:00
aoiasd
6f184d354f
enhance: Refine describe collection error message when collection not found (#30766) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-03-01 12:53:00 +08:00