test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 11:59:00 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
18,835 Commits 94 Branches 105 Tags 184 MiB
797847904c
Commit Graph

18835 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
congqixia
25a4525297
enhance: Change sync manager parallel config item (#29216) 
Since the sync manager is global in datanode now, the old
`maxParallelSyncTaskNum` does not fit into current implementation
anymore.

This PR add a new param item for sync mgr parallel control and enlarge
default value

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 20:46:41 +08:00
zhuwenxing
6efb7afd3f
test: add more request type checker for test (#29210) 
add more request type checker for test
* partition 
* database
* upsert

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-12-14 19:38:45 +08:00
congqixia
8a63e53421
enhance: Add http method to control datacoord garbage collection (#29052) 
See also #29051

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 19:26:39 +08:00
XuanYang-cn
cc727ace61
fix: Set compacted segments' level to level one (#29190) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-14 18:46:40 +08:00
yah01
13beb5ccc0
fix: load gets stuck probably (#29191) 
we found the load got stuck probably, and reviewed the logs.

the target observer seems not working, the reason is the taskDispatcher
removes the task in a goroutine, and modifies the task status after
committing the task into the goroutine pool, but this may happen after
the task removed, which leads to the task will never be removed

related #29086

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-14 18:28:38 +08:00
yah01
b8674811cf
fix: data race in ProxyClientManager (#29206) 
this PR changed the ProxyClientManager to thread-safe
fix #29205

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-14 18:22:39 +08:00
sre-ci-robot
bd640754ac
enhance: [skip e2e] skip maximize build space plugin if it is self-hosted runner (#29214) 
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
Co-authored-by: Sammy Huang <sammy.huang@zilliz.com>
 2023-12-14 17:44:39 +08:00
sre-ci-robot
3e66e78508
[automated] Update Knowhere Commit (#29178) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-14 17:16:39 +08:00
wei liu
008bae675d
enhance: Skip balance segment when channel need be balanced (#29116) 
issue: #28622
After we support balance segment with growing segment count #28623, if
we balance segment and channel at same time, some segments need to be
rebalanced after balance channel finish.

This PR skip balance segment when channel need be balanced.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-14 16:44:43 +08:00
SimFG
5a7fb97e07
fix: missing the grant info when using the SelectGrant api with the default db (#29173) 
issue: #29172

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-12-14 16:20:43 +08:00
congqixia
264505b706
enhance: remove reset seed for every shuffle (#29195) 
See also #29113
rand.Seed is deprecated and cost noticable CPU time during heavy payload

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 15:46:38 +08:00
wayblink
2274aa3b50
fix: bulkinsert binlog didn't consider ts order when processing delta data (#29163) 
#29162

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-14 14:36:40 +08:00
yah01
58dbb7872a
fix: forbid balancing level zero segments (#29168) 
related #29128
Signed-off-by: yah01 <yah2er0ne@outlook.com>

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-14 14:30:39 +08:00
Jiquan Long
f79c6146bd
enhance: [skip-e2e] auto cancel stale workflows (#29145) 
/kind improvement

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-14 11:30:38 +08:00
congqixia
9407be6eb2
enhance: Add a config item for partition name as regexp feature and disable it by default (#29154) 
See also #29177
Add a config item for partition name as regexp feature and disable it by
default

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 10:04:38 +08:00
zhuwenxing
b348827102
test: add array data type and parquet file type for bulk insert case (#29030) 
add array data type and parquet file type for the bulk insert case

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-12-13 19:56:38 +08:00
aoiasd
b5ee563914
fix: accesslog can not print search expression (#28899) 
relate: https://github.com/milvus-io/milvus/issues/28893

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-13 18:50:42 +08:00
XuanYang-cn
2c62bcbbf6
fix: Fix some left comments (#29027) 
See also: #28814

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-13 18:48:39 +08:00
Bingyi Sun
ad866d2889
feat: integrate storagev2 into index build process (#28995) 
issue: https://github.com/milvus-io/milvus/issues/28994

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-13 17:24:38 +08:00
Jiquan Long
ed79505d31
enhance: use specific rust version to compile opendal (#29106) 
/kind improvement

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-13 16:34:38 +08:00
Jiquan Long
3acb672497
[automated] Update Builder gpu image changes (#29157) 
/kind improvement
Update Builder gpu image changes
See changes:
4651f89944

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-13 14:32:37 +08:00
wei liu
fe1eeae2aa
enhance: Use mockery to replace manual mock code (#29074) 
issue: #29043
This PR remove mannul mock code for proxy and data coord

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-13 10:46:44 +08:00
sre-ci-robot
990d723496
[automated] Update Builder image changes (#29151) 
Update Builder image changes
See changes:
4651f89944
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-13 09:34:39 +08:00
nico
a1d6f02121
test: modify some cases for error msg update (#29022) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2023-12-13 08:54:40 +08:00
congqixia
cb75e73c77
enhance: Use zap.Stringer for large log field (#29143) 
See also #29113
Using zap.Stringer log field will evaluate log field value only when log
level meets the configuration, which could save some CPU time in search
route

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-12 22:52:39 +08:00
Jiquan Long
4651f89944
enhance: specify version while rust-init (#29108) 
This will reduce the size of builder image.
/kind improvement

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-12 22:10:41 +08:00
Jiquan Long
520c07ec9d
fix: autoindex panic with flat index type (#29071) 
issue: https://github.com/milvus-io/milvus/issues/29048

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-12 22:08:46 +08:00
yah01
2f0c7a6544
fix: forbid balancing level zero segments (#29130) 
we can't balance the L0 segments
related #29128

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-12 20:38:38 +08:00
PowderLi
cb5c06c718
fix: wait server to listen the http port (#29075) 
issue: #29068

wait server to listen the http port
then check whether various urls can be accessed normally

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-12 16:22:50 +08:00
cai.zhang
49b8657f95
enhance: Support implicit type conversion for parquet (#29046) 
issue: #29019

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-12 16:14:44 +08:00
Jiquan Long
e5f155612a
fix: txn control not applied to snapshot.Remove (#29124) 
issue: #29123

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-12 14:48:38 +08:00
yah01
9819090247
enhance: add more logs for target updating (#29090) 
- add more logs about the condition satisfying

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-12 14:06:43 +08:00
wayblink
51f870da7e
feat: Introduce channelCheckpointUpdater to reduce goroutine use in ttNode (#28570) 
/kind enhancement

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-12 13:48:42 +08:00
wei liu
69a71e11d1
enhance: Add config for auto upgrade segment index (#29112) 
This PR add config for auto upgrade segment index to index engine's
version.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-12 11:56:37 +08:00
Enwei Jiao
0e65e90338
enhance: Support otlp with insecure (#29115) 
issue: https://github.com/milvus-io/milvus/issues/28914

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-12-12 11:14:37 +08:00
yah01
0a87724f18
enhance: remove merger for load segments (#29062) 
remove merger as now QueryNode could load segments concurrently
fix #29063

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-12 10:48:37 +08:00
congqixia
d0bac9d0bb
enhance: Avoid initializing casbin enforcer for each request (#29117) 
See also #29113

This patch:
- Replace plain Enforcer with `casbin.SyncedEnforcer`
- Add implementation of persist.Adapter with `MetaCacheCasbinAdapter`
- Invoke enforcer.LoadPolicy when policy updated

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-12 10:36:43 +08:00
wayblink
994b239161
fix: fix datarace in root_coord proxy_client_manager (#28796) 
bug: #28797

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-12 10:18:39 +08:00
SimFG
ef18e6a532
enhance: Use a non-blocking method to trigger compaction when saveBinlogPath is executed (#28941) 
/kind improvement
issue: #28924

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-12-12 10:16:39 +08:00
congqixia
a67fc08865
fix: balance_unstable_view unit test (#29127) 
fix: #29126
Allow unstable output channel balance plan

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-12 10:02:39 +08:00
Xiaofan
9d54d6f590
fix: change Abseil to shared library to solve macos compilation issue (#28986) 
fix the compilation error on macos 14.0 with x86 arch processor
related to #28985

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2023-12-11 21:54:37 +08:00
aoiasd
3c32ba2407
enhance: pack datacoord Cluster and SessionManager with interface and mock them (#28869) 
relate: https://github.com/milvus-io/milvus/issues/28861
https://github.com/milvus-io/milvus/issues/28854

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-11 17:52:37 +08:00
congqixia
1fe5f12bd5
enhance: Add client connect wrapper to keep connection alive (#29058) 
See also #29057
Add wrapper to maintain client&connection
When reset operation is needed, `Close` method shall wait until all
on-going request return

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-11 17:20:38 +08:00
wei liu
42e538b683
enhance: enable balance channel in querycoord (#28469) 
issue: #23726

/kind improvement

1. enable auto balance channel between nodes in querycoord
2. make `genSegmentPlan` reuse the `AssignSegment` logic
3. make `genChannelPlan` reuse the `AssignChannel` logic

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-11 14:18:37 +08:00
XuanYang-cn
d2207cac27
fix: Fix rustup update (#29093) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-11 11:36:37 +08:00
Jiquan Long
02621e131c
enhance: [skip-e2e] enlarge disk space of action runner (#29077) 
/kind improvement

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-08 16:14:07 +08:00
cai.zhang
2b05460ef9
enhance: Make import-related error message clearer (#28978) 
issue: #28976

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-08 10:12:38 +08:00
MrPresent-Han
464bc9e8f4
fix: fix reduce precision for search(#27325) (#29031) 
related: #27325

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-12-08 10:04:37 +08:00
yah01
c4dda3c50f
fix: panic caused by type assert LocalSegment on Segment (#29018) 
related #29017

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-08 01:50:38 +08:00
yihao.dai
43abe9ce31
enhance: Delete datanode metric after flowgraph released (#29003) 
`DataNodeFlowGraphBufferDataSize` monitoring metric is leaked. This PR
resolve it.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-07 17:00:35 +08:00