test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 11:59:00 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
18,686 Commits 94 Branches 105 Tags 184 MiB
dd9c61831d
Commit Graph

18686 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
yihao.dai
43abe9ce31
enhance: Delete datanode metric after flowgraph released (#29003) 
`DataNodeFlowGraphBufferDataSize` monitoring metric is leaked. This PR
resolve it.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-07 17:00:35 +08:00
yah01
9b3e06ae86
enhance: add more metrics for level zero segments (#29029) 
- Add SegmentNum metric for level zero segments
- Add level zero segments size metirc

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-07 14:34:35 +08:00
Jiquan Long
52594c8596
enhance: add libasan to amazon linux runner (#29039) 
fix: #29038

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-07 14:14:42 +08:00
congqixia
dcb662d9ed
enhance: Refine C.NewSegment response and handle exception (#28952) 
See also #28795

Orignal `C.NewSegment` may panic if some condition is not met, this pr
changes response struct to `CNewSegmentResult`, which contains
`C.CStatus` and may return catched exception

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-07 13:34:35 +08:00
congqixia
cb43647b9e
enhance: Log channel checkpoint source info in writebuffer (#28993) 
See also #27675
Print channel checkpoint source with rated log will help debugging
system behavior

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-07 11:50:36 +08:00
wayblink
6736f65345
feat: skip some empty ttMsg in Datanode flowgraph (#28756) 
/kind feature

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-07 01:00:37 +08:00
cai.zhang
fb089cda8b
enhance: Load raw data while scalar index doesn't have raw data (#28888) 
issue: #28886

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-06 20:36:36 +08:00
congqixia
6a86ac0ac6
fix: Align minio object storage ut to new minio server behavior (#29014) 
See also #29013

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-06 15:42:43 +08:00
shaoyue
4a067a4c8c
enhance: Add proxy.ginLogSkipPaths (#28945) 
Fix #28944

/cc @xiaofan-luan

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-12-06 10:26:35 +08:00
XuanYang-cn
5bac7f7897
fix: Fix L0 compaction in datacoord (#28814) 
See also: #27606

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-05 18:44:37 +08:00
congqixia
cb31016640
enhance: Write buffer time range when syncing logs (#28970) 
Related to #27675
The timestamp from, to field is not field for new implementation of
writebuffer & sync manager
This pr fills these field for better log information

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-05 17:36:36 +08:00
yah01
fab52d167b
fix: may miss stream delta while loading (#28871) 
we consume the delta data from the lastest channel checkpoint while
loading segment,

this works well without level 0 segments, but now it may lead to miss
some delta data,

so we have to consume from the current target's channel checkpoint

related: #27349

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-05 17:34:45 +08:00
jaime
bb82074937
fix: Fix missing target db name for RenameCollection (#28909) 
issue: #28908

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-12-05 17:12:37 +08:00
Jiquan Long
1037b16471
[automated] Update Builder image changes (#28982) 
/kind improvement

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-05 16:52:36 +08:00
aoiasd
3cc4209d26
enhance: pack proxy connection code and support accesslog print SDK_Version (#28835) 
relate: https://github.com/milvus-io/milvus/issues/28086
https://github.com/milvus-io/milvus/issues/28940

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-05 16:50:47 +08:00
Bingyi Sun
36f69ea031
feat: integrate storagev2 in building index of segcore (#28768) 
issue: https://github.com/milvus-io/milvus/issues/28655

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-05 16:48:54 +08:00
groot
ad1daebc8e
fix: Change ranges of HNSW parameters according to Knowhere (#28875) 
fix: https://github.com/milvus-io/milvus/issues/28860

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-12-05 16:20:42 +08:00
xige-16
89b965e2b1
fix: clean snapshots meta (#28502) 
issue: https://github.com/milvus-io/milvus/issues/28496
/kind bug

The input parameters collection.partitions and collection.Field are both
nil, so these two metas have not been cleared.

Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-12-05 16:04:36 +08:00
sre-ci-robot
f01e507b15
[automated] Update Knowhere Commit (#28965) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-05 15:56:35 +08:00
sammy.huang
11687dce60
enhance: [skip e2e]upgrade minio version (#28971) 
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
 2023-12-05 14:14:34 +08:00
yihao.dai
7f819ac594
enhance: Remove partition load infos with prefix (#28969) 
Removing partition load infos with prefix instead of removing them one
by one. This enhance can save transactions.

issue: https://github.com/milvus-io/milvus/issues/28343

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-05 12:52:34 +08:00
Jiquan Long
e2d0333831
[automated] Update Builder gpu image changes (#28963) 
/kind improvement

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-05 11:58:33 +08:00
Bingyi Sun
10bb2431d8
test: add checker unittests (#28954) 
issue: https://github.com/milvus-io/milvus/issues/28610

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-05 10:56:33 +08:00
sre-ci-robot
91e92ee8d4
[automated] Update Pytest image changes (#28938) 
Update Pytest image changes
See changes:
8740adbe28
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-05 10:06:35 +08:00
sre-ci-robot
ab630548f4 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2023-12-04 12:48:33 +00:00
yihao.dai
d26b563a8b
feat: Define import API and metadata (#28731) 
Define the new rpc and metadata for ImportV2.

see also: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-04 19:56:35 +08:00
congqixia
393b1f943c
fix: Reject compaction task with growing segments (#28925) 
See also #28924
The compaction task generated before datanode finish `SaveBinlogPath`
grpc call contains segments which are still in Growing state DataNode
shall verify each non-levelzero segments before submit compaction task
to executor

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-04 19:06:40 +08:00
congqixia
e79c3eaa90
enhance: [skip e2e] Update install_deps.sh cmake version (#28946) 
cmake version needs to be greater or equal than 3.26.4, sync
`install_deps.sh` scripts version to 3.26.5

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-04 18:26:34 +08:00
aoiasd
b4af6f8c40
fix: sync action load segment with lack collection index info list (#28788) 
relate: https://github.com/milvus-io/milvus/issues/28779
https://github.com/milvus-io/milvus/issues/28637

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-04 18:14:34 +08:00
Bingyi Sun
45e6801ce4
feat: Add checker activation service interfaces (#28850) 
issue: #28610

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-04 17:38:37 +08:00
sre-ci-robot
9b6cbe956a
[automated] Update Knowhere Commit (#28917) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-04 15:42:34 +08:00
yihao.dai
9723e16058
enhance: Update pymilvus (#28922) 
issue: https://github.com/milvus-io/milvus/issues/17614

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-04 15:36:35 +08:00
congqixia
c8b1a4618a
enhance: Resolve libunwind requirement conflict using 1.7.2 (#28929) 
Try to resolve libunwind dependency requirement conflict between glog &
folly

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-04 14:06:40 +08:00
nico
cbd3b3c836
test: modify test cases for error msg update (#28846) 
#23075

Signed-off-by: nico <cheng.yuan@zilliz.com>
 2023-12-04 11:22:37 +08:00
nico
8740adbe28
test: update pymilvus version (#28832) 
#17614

Signed-off-by: nico <cheng.yuan@zilliz.com>
 2023-12-04 11:20:47 +08:00
wei liu
d081fd5481
enhance: Change some frequency log to rated level (#28897) 
This pr change some frequency log's level to rated.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-04 10:38:35 +08:00
Jiquan Long
fccec125b2
enhance: [skip-e2e] add rust to builder image (#28838) 
https://github.com/milvus-io/milvus/issues/27704

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-04 10:10:35 +08:00
wei liu
6718986a5b
enhance: Remove too frequency read ddl log from rootcoord (#28891) 
This PR removed too frequency log for such
`DescribeCollection/ShowPartition` operation from root coord

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-04 09:42:33 +08:00
PowderLi
20fc90c591
enhance: find collection schema from cache (#28782) 
issue: #28781 #28329

1. There is no need to call `DescribeCollection`, if the collection's
schema is found in the globalMetaCache
2. did `GetProperties` to check the access to Azure Blob Service while
construct the ChunkManager

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-03 19:22:33 +08:00
yah01
342635ed61
enhance: enable assert method to format arguments (#28812) 
for now the assert method in segcore could accept a string information,
too many codes don't print the value they assert.

make it happy
related #28811

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-01 18:04:33 +08:00
aoiasd
24c565e37b
fix: accesslog method status not return failed when error in response (#28824) 
relate: https://github.com/milvus-io/milvus/issues/28086

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-01 15:30:38 +08:00
sammy.huang
1174a7527a
enhance: [skip e2e]delete unsed arch for gpu builder (#28887) 
1. disable arm64 arch 
2. only use docker cli instead of docker buildx cli

---------

Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
 2023-12-01 14:48:32 +08:00
yihao.dai
f5856812a2
fix: Fix get binary vector from chunk cache (#28866) 
The way of getting binary vector size is wrong. This PR will fix it.

issue: https://github.com/milvus-io/milvus/issues/28865

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-01 14:40:32 +08:00
Gao
7206795e91
fix: update folly to resolve simd issue (#28878) 
related #27552 , after this, milvus could run successfully on sse4.2
only machine

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-12-01 13:50:32 +08:00
congqixia
038eebba4d
enhance: Move datacoord broker into separate package (#28876) 
See also #28861

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-01 10:22:34 +08:00
Bingyi Sun
8036ee13fa
feat: avoid dereferencing nullptr (#28862) 
issue: #28793

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-01 10:20:32 +08:00
wei liu
043ac87be0
fix: Balance channel may cause channel not availble error (#28829) 
issue: #28831
release old delegator before new delegator update it's distribution may
cause `channel not availble` error

This PR will block release old delgator before new delegator finish
`syncDistribution`

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-01 10:08:34 +08:00
sre-ci-robot
af98d1cb64
[automated] Update Builder image changes (#28874) 
Update Builder image changes
See changes:
cac802ef7f
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-01 10:04:32 +08:00
sre-ci-robot
ecc3ca374c
[automated] Update Knowhere Commit (#28882) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-01 02:28:31 +08:00
XuanYang-cn
e62edb991a
enhance: Add FlowgraphManager interface (#28852) 
- Change flowgraphManager to fgManagerImpl
- Change close to stop
- change execute to controlMemWaterLevel
- Change method name of fgManager for readability
- Add mockery for fgmanager

Issue: #28853

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-30 18:42:32 +08:00