congqixia
b70ede4c9b
enhance: [2.4] Set WatchProgress for channel operation progress response ( #35183 ) ( #35231 )
...
Cherry-pick from master
pr: #35183
The watch progress is always zero in CheckChannelOperationProgress
response, which is meaningless and confusing. This PR set progress value
in rpc response to fix this problem.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-02 16:52:15 +08:00
wayblink
fc974c4db1
enhance: [cherry-pick] improve ut for compaction_policy_clustering ( #35206 )
...
#34792
master pr: #35205
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-08-02 16:38:15 +08:00
wei liu
d767f8977a
enhance: Refine param init for MmapDirPath ( #35181 ) ( #35214 )
...
pr: #35181
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-02 16:30:15 +08:00
congqixia
f8444b900f
enhance: [2.4] Support proxy/delegator qn client pooling ( #35195 )
...
Cherry pick from master
pr: #35194
See also #35196
Add param item for proxy/delegator query node client pooling and
implement pooling logic
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-02 11:24:19 +08:00
wayblink
8137c26330
enhance: [cherry-pick] unify time in clustering compaction task to unix ( #35168 )
...
related: #34495
pr: #35167
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-08-02 10:30:19 +08:00
Jiquan Long
ae3657b23e
enhance: add metric to record maxInsertRate and queryNodeMemoryHighWaterLevel ( #35185 ) ( #35188 )
...
pr: https://github.com/milvus-io/milvus/pull/35185
fix: https://github.com/milvus-io/milvus/issues/35177
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-02 10:16:15 +08:00
cai.zhang
5f3d41d6c6
enhance:[cherry-pick] Support calculate segment maxRows with diskann ( #35155 )
...
issue: #34495
master pr: #35076
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-02 09:56:14 +08:00
cai.zhang
f7184101e1
fix: [cherry-pick] Fix data race for clustering buffer writer ( #35146 )
...
issue: #34495
master pr: #35145
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-01 16:08:13 +08:00
wei liu
5f601fcc50
enhance: Reduce delegator memory overloaded factor to 0.1 ( #35092 ) ( #35164 )
...
pr: #35092
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-01 14:20:13 +08:00
nico
9bebca472b
test: update pymilvus version ( #35143 )
...
pr: #34843
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-08-01 11:04:13 +08:00
congqixia
824b26c209
enhance: [2.4] Pre-allocate space for reduce data structure ( #35118 ) ( #35137 )
...
Cherry-pick from master
pr: #35118
Grow slice & map.growWork may cause a lot when segment number is large
for big K query. This PR pre-allocate space for reduce methods to avoid
this cost.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-01 09:59:50 +08:00
wayblink
37a5f91683
enhance: [cherry-pick] add commit time in partitionStats proto ( #35124 )
...
fix : #35110
pr: #35125
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-31 21:22:04 +08:00
Gao
be0123863f
enhance: add channel num for queryHook optimization ( #35105 )
...
pr: #35104
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-07-31 18:23:51 +08:00
wei liu
f8f2736cda
enhance: Skip manual stopped component during health check ( #34953 ) ( #35123 )
...
pr: #34953
after manual stop component by management restful api, `healthz` may
return unhealthy state. k8s may restart the pod to save the unhealthy
sate, and the manual stop operation will got unexpected result.
to solve this, we make `healthz` API skip the manual stopped component.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-31 18:01:51 +08:00
Patrick Weizhi Xu
85d8200acb
enhance: [2.4] improve error msg when expr invalid during isolation ( #35115 )
...
issue: #34336
pr: #35127
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-07-31 17:21:50 +08:00
XuanYang-cn
cf56a1452c
enhance: Add more ut for l0CompactionTask ( #35101 )
...
See also: #34796
pr: #35100
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-07-31 17:19:50 +08:00
Patrick Weizhi Xu
0bec383342
fix: [2.4] disable sparse and binary in MV ( #35121 )
...
issue: #35120
pr: #35126
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-07-31 15:02:01 +08:00
congqixia
f5d0e55ebe
enhance: [2.4] Resolve conflict cpp format ( #35129 )
...
See also #35073 #35119
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-31 12:15:50 +08:00
congqixia
94c86327ae
enhance: [2.4] Use clang-format-12 to perform cpp check ( #35119 )
...
clang-format version is 12 after upgrading to ubuntu 22.04, this PR
change formating script to use clang-format-12 and fix existing problems
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-31 10:33:50 +08:00
cai.zhang
c340f387cf
enhance: [cherry-pick] Change the fixed value to a ratio for clustering segment size ( #35075 )
...
issue: #34495
master pr: #35076
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-31 10:32:00 +08:00
cai.zhang
4bbb7b8219
enhance: [cherry-pick] Skip updating the level that are already at the specified level ( #35093 )
...
issue: #34495
master pr: #35094
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-31 09:39:54 +08:00
congqixia
935a117396
enhance: [2.4] Support otlp http exporter ( #35053 ) ( #35073 )
...
Cherry-pick from master
pr: #35053
See also #35052
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 21:00:08 +08:00
congqixia
8991dc211e
enhance: [2.4] Fix go&cpp lint issues ( #35107 )
...
See also #34483
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 20:25:55 +08:00
zhagnlu
866055527b
enhance: revert remove duplicated pk function ( #35102 )
...
pr: #35103
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-30 20:03:50 +08:00
foxspy
410eb8a787
fix: [2.4] add prometheus dependency for milvus_monitor ( #35079 )
...
issue: #35077
pr: #35078
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-07-30 19:41:49 +08:00
Jiquan Long
86edca8c1b
fix: support auto index for array ( #35095 )
...
/kind branch-feature
pr: #34450
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: Zhagnlu <lu.zhang@zilliz.com>
2024-07-30 17:57:50 +08:00
Congqi Xia
d51bfcdd7f
Skip cases
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 16:10:38 +08:00
PowderLi
fa9d606aa8
enhance: add metrics for restful apis ( #35083 )
...
issue: #34968
pr: #34969
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-07-30 14:13:49 +08:00
chyezh
b6c1aeb6d9
enhance: the datacoord gc should fast quitable ( #35051 )
...
issue: #35049
pr: #35050
Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-30 14:11:58 +08:00
yihao.dai
cc188fb322
enhance: Avoid panic due to nil schema ( #35063 ) ( #35064 )
...
/kind improvement
issue: https://github.com/milvus-io/milvus/discussions/25620
pr: https://github.com/milvus-io/milvus/pull/35063
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-29 20:07:49 +08:00
yellow-shine
4138a9f87d
enhance: [skip e2e] enlarge timeout of UT for go up to 90 mins ( #35037 )
...
pr: #35035
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-07-29 15:53:58 +08:00
Patrick Weizhi Xu
e2fc8853b6
fix: [2.4] disallow expr when partition key isolation is enabled ( #35025 )
...
issue: #34336
pr: #35031
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-07-29 14:23:49 +08:00
wei liu
a2c19af3ed
enhance: Solve channel unbalance on datanode ( #34984 ) ( #35033 )
...
issue: #33583
pr: #34984
the old policy permit datanode has at most 2 more channels than other
datanode. so if milvus has 2 datanode and 2 channels, both 2 channels
will be assign to 1 datanode, left another datanode empty.
This PR refine the balance policy to solve channel unbalance on datanode
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:33:50 +08:00
congqixia
d16320705e
enhance: [2.4] Add Segment Level in milvus segment info APIs ( #34763 ) ( #35023 )
...
Cherry-pick from master
pr: #34763
See also #34746
This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`
---------
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-29 10:11:52 +08:00
shaoyue
0417463e90
enhance: Gin & restfulv1 handler use sonic json ( #35018 )
...
/cc @czs007 @congqixia
master pr: #35020
Signed-off-by: haorenfsa <haorenfsa@gmail.com>
2024-07-29 00:39:49 +08:00
zhagnlu
2e9a86ca80
enhance: redefine variable column block size ( #35041 )
...
pr: #35040
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-28 23:49:48 +08:00
congqixia
079276c6ff
fix: [2.4] Unify hook singleton implementation in proxy ( #34888 )
...
Cherry-pick from master
pr: #34887
Related to #34885
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 18:07:53 +08:00
wayblink
1e5c71d550
fix: [cherry-pick] fix dropped segment still visible after dropped by L2 single compaction ( #35006 )
...
bug: #35003
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-26 13:47:48 +08:00
congqixia
2a43f43916
fix: [2.4] Remove timeout in datanode watch ctx ( #35011 ) ( #35017 )
...
Cherry-pick from master
pr: #35011
See also #35008
Use tickle timeout logic instead of hardcode context timeout
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 11:59:46 +08:00
cai.zhang
9cd6dbcbc9
fix: [cherry-pick] Fix bug for block clustering compaction ( #35021 )
...
issue: #34703
master pr: #35019
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-26 11:33:40 +08:00
zhagnlu
6f0f3034c6
fix: fix query count(*) concurrently ( #35010 )
...
pr: #35007
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-26 11:11:46 +08:00
wei liu
b3bc7f3985
enhance: Limit collection's normal balance speed ( #34810 ) ( #34987 )
...
issue: #34798
pr: #34810
after we remove the task priority on query coord, to avoid load/release
segment blocked by too much balance task, we limit the balance task size
in each round. at same time, we reduce the balance interval to trigger
balance more frequently.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-26 10:13:46 +08:00
zhagnlu
99da94ee32
enhance: decrease variable column block size ( #35012 )
...
#35013
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-25 21:54:08 +08:00
zhagnlu
9eb42bbda0
fix: fix parse plan proto failed for search type ( #34957 )
...
pr: #34944
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-25 20:27:46 +08:00
wayblink
8c87ec0bc5
enhance: [cherry-pick] refine error handle in clustering compaction task ( #34958 )
...
related: #30633 #34792
pr: #34956
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-25 19:25:46 +08:00
Jiquan Long
2822d8729c
feat: record the number of times milvus turns into force-deny-writing states ( #34960 ) ( #34989 )
...
/kind improvement
fix : #34961
pr: #34960
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-07-25 15:43:09 +08:00
Chun Han
6283fd0b46
fix:nil part stats without l2 compaction ( #34977 )
...
related: #34923
pr: https://github.com/milvus-io/milvus/pull/34992
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-25 14:21:46 +08:00
congqixia
0dd9af8f13
fix: [2.4] Check legacy guarantee ts when skipping alloc ts ( #34982 )
...
Cherry-pick from master
pr: #34981
See also #34980
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-25 10:27:47 +08:00
jaime
77ae127a62
fix: check collection health(queryable) fail for releasing collection ( #34948 )
...
issue: #34946
pr: #34947
---------
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-25 10:25:57 +08:00
cai.zhang
12a24c3957
enhance: [cherry-pick] Refine code for GetRecoveryInfo ( #34974 )
...
issue: #34495
master pr: #34973
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-24 22:01:44 +08:00
