sammy.huang
9083f6999d
enhance: bind dependency of miluvs during e2e to dedicated nodes ( #32600 )
...
#32627
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-26 11:43:25 +08:00
shaoting-huang
f5e1fc757c
fix: [skip e2e] dev container yaml indentation error ( #32601 )
...
issue: #32594
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-04-26 10:15:27 +08:00
chyezh
2586c2f1b3
enhance: use WalkWithPrefix api for oss, enable piplined file gc ( #31740 )
...
issue: #19095,#29655,#31718
- Change `ListWithPrefix` to `WalkWithPrefix` of OOS into a pipeline
mode.
- File garbage collection is performed in other goroutine.
- Segment Index Recycle clean index file too.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-25 20:41:27 +08:00
chyezh
f06509bf97
fix: get replica should not report error when no querynode serve ( #32536 )
...
issue: #30647
- Remove error report if there's no query node serve. It's hard for
programer to use it to do resource management.
- Change resource group `transferNode` logic to keep compatible with old
version sdk.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-25 19:25:24 +08:00
congqixia
0ff7a46e95
fix: [skip e2e] Disable compaction for balance integration test ( #32603 )
...
See also #31468
Balance test suite may assert segment number based on test setup.
However the compaction may reduce the number and cause test cases
unstable.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-25 16:55:23 +08:00
congqixia
076b8f2fbc
doc: [skip e2e] Add document for developing with local milvus-proto guide ( #31513 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-25 16:35:25 +08:00
Buqian Zheng
8a1017a152
enhance: add helpers to parse sparse float vector in JSON ( #32543 )
...
issue: #29419
added helper functions to parse JSON representation of sparse float
vectors, will be used by both the restful server and the import utils.
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-04-25 14:47:24 +08:00
chyezh
b287fbaa2e
fix: return collection on recovering but not collection not loaded when target is not recovered ( #32447 )
...
issue: #32398
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-25 11:21:26 +08:00
congqixia
f30c22626e
enhance: Pre-cache result for frequent filters ( #32580 )
...
See also #32165
Add segment dist and leader view filter criterion struct to store
frequent filter conditions.
Add collection/channel filter results for these two meta
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-25 11:13:25 +08:00
wei liu
04f355a802
enhance: Enable alter database props in rootcoord ( #32458 )
...
issue: #30040
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-25 10:53:25 +08:00
Jiquan Long
c002745902
enhance: retrieve output fields after local reduce ( #32346 )
...
issue: #31822
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-04-25 09:49:26 +08:00
binbin
5119292411
test: update pymilvus version and modify test cases ( #32572 )
...
issue: #32376
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-04-25 08:57:31 +08:00
sre-ci-robot
b703d6cdbe
[automated] Update Knowhere Commit ( #32586 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-04-25 02:27:24 +08:00
sammy.huang
4c7d3dade3
enhance: bind e2e to dedicated nodes ( #32577 )
...
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-24 19:27:24 +08:00
congqixia
faa559592d
enhance: Make applyDelete work in paralell in segment level ( #32291 )
...
`applyDelete` used to be serial for delete entries on each segments.
This PR make it work in parallel with errgroup to improve performance
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-24 17:01:24 +08:00
congqixia
37ca32dbba
enhance: Make SegmentDistManager filter use node index ( #32533 )
...
See also #32165
Change `SegmentDistFilter` to interface in order to provde node index
when filter segment dist.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-24 16:53:24 +08:00
yihao.dai
281a583eda
fix: Correct the negative queryable num entities metric ( #32361 )
...
issue: https://github.com/milvus-io/milvus/issues/32281
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-24 15:55:24 +08:00
XuanYang-cn
15b989bb80
fix: Zero flushReq metric for all sealed segs ( #32404 )
...
See also: #32399
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-04-24 14:27:23 +08:00
congqixia
dcc15e3ed8
enhance: Remove TestRateLimiter/tests_refresh_rate_by_config case ( #32545 )
...
See also #32313
This case always fail since the watch config logic is removed
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-24 10:13:24 +08:00
PowderLi
29922829da
fix: [restful v2] parameters of create collection ( #32485 )
...
issue: #31176
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-24 09:43:29 +08:00
sre-ci-robot
96911489a6
[automated] Update gpu Builder image changes ( #32051 )
...
Update gpu Builder image changes
See changes:
08bfb431b7sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-04-23 18:53:23 +08:00
smellthemoon
365e50b63e
fix: revert add range search params check in proxy ( #32366 )
...
no need to check params in empty segment.
#30365
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-04-23 17:41:23 +08:00
congqixia
6ef677f79e
fix: Remove metrics after flowgraph closed ( #32515 )
...
See also #32403
`fg_buffer_size` was decreased after metrics removed in flowgraph
ddnode, which make metrics value negative.
This PR move remove metrics logic into `dataSyncService.Close`
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-23 17:33:23 +08:00
smellthemoon
96d95e7743
enhance: fix pass error msg as channel name ( #32511 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-04-23 16:45:22 +08:00
yihao.dai
3119a7b23f
fix: Remove watching config event in rate limiter ( #32313 )
...
Remove watching config event in rate limiter to prevent object leaks.
Instead, reset limiter rates periodically.
issue: https://github.com/milvus-io/milvus/issues/32312
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-23 16:19:24 +08:00
Cai Yudong
16b8b7b35d
enhance: Add get_vector unittest for float16 & bfloat16 ( #32153 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-04-23 16:15:23 +08:00
zhuwenxing
a5f0fc4373
test: add bf/f16 bulk insert testcase ( #32506 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-23 14:05:23 +08:00
Ted Xu
fef7812254
enhance: trigger compaction by size ( #32326 )
...
See #32328
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-04-23 12:11:24 +08:00
congqixia
bfebdecf3e
enhance: Make LeaderView Manager filter use map index ( #32505 )
...
See also #32165
Change `LeaderViewFilter` to interface to provided map key to avoid
iterating all key-values in LeaderViewManager
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-23 11:07:24 +08:00
chyezh
21a9de5c8e
fix: resource group ut fixup ( #32509 )
...
issue: #30647
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-23 10:01:23 +08:00
pingliu
72b054e0c3
doc: change milvus docker image version to v2.4.0 ( #32489 )
...
change milvus docker image version to v2.4.0
Signed-off-by: ping.liu <ping.liu@zilliz.com>
2024-04-23 09:47:23 +08:00
sre-ci-robot
15558112c5
[automated] Update Knowhere Commit ( #32516 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-04-23 01:33:23 +08:00
sre-ci-robot
ad5a49e6cc
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-04-22 12:00:56 +00:00
Gao
0fab265eed
enhance: update knowhere and some header changes ( #32468 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-04-22 15:47:26 +08:00
aoiasd
d0ffb71a17
enhance: forbid delete with always true expression ( #32472 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-04-22 11:09:23 +08:00
congqixia
d7ff1bbe5c
enhance: Make querycoordv2 collection observer task driven ( #32441 )
...
See also #32440
- Add loadTask in collection observer
- For load collection/partitions, load task shall timeout as a whole
- Change related constructor to load jobs
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-22 10:39:22 +08:00
zhuwenxing
8442098457
test: skip index if it already created ( #32402 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-22 10:37:22 +08:00
Cai Yudong
5fc439c600
feat: Bulk insert support fp16/bf16 ( #32157 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-04-22 10:05:22 +08:00
yiwangdr
037de8e4d3
enhance: speed up minor functions calls in datacoord ( #32389 )
...
Related to https://github.com/milvus-io/milvus/issues/32165
1. nodeid based channel store access should use map access instead of
iteration.
2. The join-ish functions calls are slow when # collections/segments
increases (e.g. 10k).
e.g.
getNumRowsOfCollectionUnsafe is O(num_segments); GetAllCollectionNumRows
is of O(num_collections*num_segments).
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-04-20 07:55:21 +08:00
zhagnlu
93f0c262c1
fix: add bitset sve support macro protect ( #32463 )
...
#32137
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-04-19 18:05:21 +08:00
Ted Xu
744a54a534
enhance: enforce milvus.yaml assertion in UT ( #32357 )
...
See #32168
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-04-19 16:47:20 +08:00
XuanYang-cn
7e830f6185
fix: [skip-e2e] TestAvgReassignPolicy ut ( #32450 )
...
See also: #32448
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-04-19 15:11:20 +08:00
congqixia
01c16fe6e3
enhance: Manual release pool after save targets ( #32358 )
...
See also #31632
Release conc.Pool after usage to clean worker and stop background purge
and ticktock.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-19 13:51:21 +08:00
SimFG
8594b55ad5
enhance: add max insert request size and must use partition key configs ( #32433 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:31:20 +08:00
SimFG
31a29a2451
enhance: avoid the repeated metric info in the proxy ( #32380 )
...
issue: #30577
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:21:20 +08:00
jaime
18c3cbe46d
enhance: Update tls user doc ( #32353 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-04-18 21:07:23 +08:00
zhuwenxing
109cd0402f
test: fix delete expr ( #32377 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-18 17:25:20 +08:00
zhuwenxing
2cff65511c
test: use float32 to do import ( #32276 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-18 17:23:27 +08:00
sammy.huang
2a4dfde2de
enhance: revert [skip e2e] use pull_request_target event instead ( #32426 )
...
This reverts commit ab6ddf6929
2024-04-18 17:15:37 +08:00
nico
e83651def4
test: update test cases ( #32396 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-04-18 16:15:25 +08:00
