test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 20:09:57 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
18,608 Commits 94 Branches 105 Tags 184 MiB
2f0c7a6544
Commit Graph

18608 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
yah01
2f0c7a6544
fix: forbid balancing level zero segments (#29130) 
we can't balance the L0 segments
related #29128

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-12 20:38:38 +08:00
PowderLi
cb5c06c718
fix: wait server to listen the http port (#29075) 
issue: #29068

wait server to listen the http port
then check whether various urls can be accessed normally

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-12 16:22:50 +08:00
cai.zhang
49b8657f95
enhance: Support implicit type conversion for parquet (#29046) 
issue: #29019

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-12 16:14:44 +08:00
Jiquan Long
e5f155612a
fix: txn control not applied to snapshot.Remove (#29124) 
issue: #29123

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-12 14:48:38 +08:00
yah01
9819090247
enhance: add more logs for target updating (#29090) 
- add more logs about the condition satisfying

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-12 14:06:43 +08:00
wayblink
51f870da7e
feat: Introduce channelCheckpointUpdater to reduce goroutine use in ttNode (#28570) 
/kind enhancement

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-12 13:48:42 +08:00
wei liu
69a71e11d1
enhance: Add config for auto upgrade segment index (#29112) 
This PR add config for auto upgrade segment index to index engine's
version.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-12 11:56:37 +08:00
Enwei Jiao
0e65e90338
enhance: Support otlp with insecure (#29115) 
issue: https://github.com/milvus-io/milvus/issues/28914

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-12-12 11:14:37 +08:00
yah01
0a87724f18
enhance: remove merger for load segments (#29062) 
remove merger as now QueryNode could load segments concurrently
fix #29063

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-12 10:48:37 +08:00
congqixia
d0bac9d0bb
enhance: Avoid initializing casbin enforcer for each request (#29117) 
See also #29113

This patch:
- Replace plain Enforcer with `casbin.SyncedEnforcer`
- Add implementation of persist.Adapter with `MetaCacheCasbinAdapter`
- Invoke enforcer.LoadPolicy when policy updated

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-12 10:36:43 +08:00
wayblink
994b239161
fix: fix datarace in root_coord proxy_client_manager (#28796) 
bug: #28797

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-12 10:18:39 +08:00
SimFG
ef18e6a532
enhance: Use a non-blocking method to trigger compaction when saveBinlogPath is executed (#28941) 
/kind improvement
issue: #28924

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-12-12 10:16:39 +08:00
congqixia
a67fc08865
fix: balance_unstable_view unit test (#29127) 
fix: #29126
Allow unstable output channel balance plan

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-12 10:02:39 +08:00
Xiaofan
9d54d6f590
fix: change Abseil to shared library to solve macos compilation issue (#28986) 
fix the compilation error on macos 14.0 with x86 arch processor
related to #28985

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2023-12-11 21:54:37 +08:00
aoiasd
3c32ba2407
enhance: pack datacoord Cluster and SessionManager with interface and mock them (#28869) 
relate: https://github.com/milvus-io/milvus/issues/28861
https://github.com/milvus-io/milvus/issues/28854

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-11 17:52:37 +08:00
congqixia
1fe5f12bd5
enhance: Add client connect wrapper to keep connection alive (#29058) 
See also #29057
Add wrapper to maintain client&connection
When reset operation is needed, `Close` method shall wait until all
on-going request return

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-11 17:20:38 +08:00
wei liu
42e538b683
enhance: enable balance channel in querycoord (#28469) 
issue: #23726

/kind improvement

1. enable auto balance channel between nodes in querycoord
2. make `genSegmentPlan` reuse the `AssignSegment` logic
3. make `genChannelPlan` reuse the `AssignChannel` logic

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-11 14:18:37 +08:00
XuanYang-cn
d2207cac27
fix: Fix rustup update (#29093) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-11 11:36:37 +08:00
Jiquan Long
02621e131c
enhance: [skip-e2e] enlarge disk space of action runner (#29077) 
/kind improvement

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-08 16:14:07 +08:00
cai.zhang
2b05460ef9
enhance: Make import-related error message clearer (#28978) 
issue: #28976

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-08 10:12:38 +08:00
MrPresent-Han
464bc9e8f4
fix: fix reduce precision for search(#27325) (#29031) 
related: #27325

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-12-08 10:04:37 +08:00
yah01
c4dda3c50f
fix: panic caused by type assert LocalSegment on Segment (#29018) 
related #29017

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-08 01:50:38 +08:00
yihao.dai
43abe9ce31
enhance: Delete datanode metric after flowgraph released (#29003) 
`DataNodeFlowGraphBufferDataSize` monitoring metric is leaked. This PR
resolve it.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-07 17:00:35 +08:00
yah01
9b3e06ae86
enhance: add more metrics for level zero segments (#29029) 
- Add SegmentNum metric for level zero segments
- Add level zero segments size metirc

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-07 14:34:35 +08:00
Jiquan Long
52594c8596
enhance: add libasan to amazon linux runner (#29039) 
fix: #29038

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-07 14:14:42 +08:00
congqixia
dcb662d9ed
enhance: Refine C.NewSegment response and handle exception (#28952) 
See also #28795

Orignal `C.NewSegment` may panic if some condition is not met, this pr
changes response struct to `CNewSegmentResult`, which contains
`C.CStatus` and may return catched exception

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-07 13:34:35 +08:00
congqixia
cb43647b9e
enhance: Log channel checkpoint source info in writebuffer (#28993) 
See also #27675
Print channel checkpoint source with rated log will help debugging
system behavior

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-07 11:50:36 +08:00
wayblink
6736f65345
feat: skip some empty ttMsg in Datanode flowgraph (#28756) 
/kind feature

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-07 01:00:37 +08:00
cai.zhang
fb089cda8b
enhance: Load raw data while scalar index doesn't have raw data (#28888) 
issue: #28886

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-06 20:36:36 +08:00
congqixia
6a86ac0ac6
fix: Align minio object storage ut to new minio server behavior (#29014) 
See also #29013

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-06 15:42:43 +08:00
shaoyue
4a067a4c8c
enhance: Add proxy.ginLogSkipPaths (#28945) 
Fix #28944

/cc @xiaofan-luan

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-12-06 10:26:35 +08:00
XuanYang-cn
5bac7f7897
fix: Fix L0 compaction in datacoord (#28814) 
See also: #27606

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-05 18:44:37 +08:00
congqixia
cb31016640
enhance: Write buffer time range when syncing logs (#28970) 
Related to #27675
The timestamp from, to field is not field for new implementation of
writebuffer & sync manager
This pr fills these field for better log information

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-05 17:36:36 +08:00
yah01
fab52d167b
fix: may miss stream delta while loading (#28871) 
we consume the delta data from the lastest channel checkpoint while
loading segment,

this works well without level 0 segments, but now it may lead to miss
some delta data,

so we have to consume from the current target's channel checkpoint

related: #27349

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-05 17:34:45 +08:00
jaime
bb82074937
fix: Fix missing target db name for RenameCollection (#28909) 
issue: #28908

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-12-05 17:12:37 +08:00
Jiquan Long
1037b16471
[automated] Update Builder image changes (#28982) 
/kind improvement

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-05 16:52:36 +08:00
aoiasd
3cc4209d26
enhance: pack proxy connection code and support accesslog print SDK_Version (#28835) 
relate: https://github.com/milvus-io/milvus/issues/28086
https://github.com/milvus-io/milvus/issues/28940

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-05 16:50:47 +08:00
Bingyi Sun
36f69ea031
feat: integrate storagev2 in building index of segcore (#28768) 
issue: https://github.com/milvus-io/milvus/issues/28655

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-05 16:48:54 +08:00
groot
ad1daebc8e
fix: Change ranges of HNSW parameters according to Knowhere (#28875) 
fix: https://github.com/milvus-io/milvus/issues/28860

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-12-05 16:20:42 +08:00
xige-16
89b965e2b1
fix: clean snapshots meta (#28502) 
issue: https://github.com/milvus-io/milvus/issues/28496
/kind bug

The input parameters collection.partitions and collection.Field are both
nil, so these two metas have not been cleared.

Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-12-05 16:04:36 +08:00
sre-ci-robot
f01e507b15
[automated] Update Knowhere Commit (#28965) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-05 15:56:35 +08:00
sammy.huang
11687dce60
enhance: [skip e2e]upgrade minio version (#28971) 
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
 2023-12-05 14:14:34 +08:00
yihao.dai
7f819ac594
enhance: Remove partition load infos with prefix (#28969) 
Removing partition load infos with prefix instead of removing them one
by one. This enhance can save transactions.

issue: https://github.com/milvus-io/milvus/issues/28343

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-05 12:52:34 +08:00
Jiquan Long
e2d0333831
[automated] Update Builder gpu image changes (#28963) 
/kind improvement

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-05 11:58:33 +08:00
Bingyi Sun
10bb2431d8
test: add checker unittests (#28954) 
issue: https://github.com/milvus-io/milvus/issues/28610

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-05 10:56:33 +08:00
sre-ci-robot
91e92ee8d4
[automated] Update Pytest image changes (#28938) 
Update Pytest image changes
See changes:
8740adbe28
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-05 10:06:35 +08:00
sre-ci-robot
ab630548f4 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2023-12-04 12:48:33 +00:00
yihao.dai
d26b563a8b
feat: Define import API and metadata (#28731) 
Define the new rpc and metadata for ImportV2.

see also: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-04 19:56:35 +08:00
congqixia
393b1f943c
fix: Reject compaction task with growing segments (#28925) 
See also #28924
The compaction task generated before datanode finish `SaveBinlogPath`
grpc call contains segments which are still in Growing state DataNode
shall verify each non-levelzero segments before submit compaction task
to executor

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-04 19:06:40 +08:00
congqixia
e79c3eaa90
enhance: [skip e2e] Update install_deps.sh cmake version (#28946) 
cmake version needs to be greater or equal than 3.26.4, sync
`install_deps.sh` scripts version to 3.26.5

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-04 18:26:34 +08:00