Ted Xu
71adafa933
enhance: adding a streaming deserialize reader for binlogs ( #30860 )
...
See #30863
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-03-04 19:31:09 +08:00
jaime
4b0c3dd377
enhance: index meta use independent rather than global meta lock ( #30869 )
...
issue: https://github.com/milvus-io/milvus/issues/30837
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-03-04 16:56:59 +08:00
sre-ci-robot
3dc5e38240
[automated] Update Knowhere Commit ( #30989 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-03-04 16:34:59 +08:00
wei liu
fa73520b57
fix: Make datacoord client retry on index api ( #30654 )
...
issue: #20553
This PR add retry on all interface which belong to indexcoord in milvus
2.2 and. move to data coord in milvus 2.3, to prevent meet
`unimplemented` error during rolling upgrade from milvus 2.2 to 2.3.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-04 15:32:59 +08:00
sre-ci-robot
882b50c5c1
[automated] Update Pytest image changes ( #30852 )
...
Update Pytest image changes
See changes:
7f78e9d40dsre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-03-04 09:55:00 +08:00
cai.zhang
f6ff2588cd
enhance: Optimize DescribeIndex to reduce lock contention ( #30939 )
...
issue: #29313
issue: #30443
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-03 19:00:59 +08:00
PowderLi
e39f46aa38
fix: use GetBinaryField incorrectly ( #30969 )
...
issue: #30980 #22837
2.3 pr: #30873
should use GetBFloat16Field, GetBfloat16Field rather than GetBinaryField
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-03-03 17:58:59 +08:00
zhuwenxing
d98a5e441f
test: add restful v2 test ( #30984 )
...
add restful v2 test
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-03-02 11:03:03 +08:00
nico
d8164c43d2
test: update test cases ( #30777 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-03-02 11:01:13 +08:00
yihao.dai
0a2c255630
enhance: Reduce the memory usage of the timeTickSender ( #30968 )
...
In the cache of the timeTickSender, retain only the latest stats instead
of storing stats for every time tick.
issue: https://github.com/milvus-io/milvus/issues/30967
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-02 10:13:01 +08:00
MrPresent-Han
29f44f840a
enhance: refine groupBy error msg( #29968 ) ( #30920 )
...
related: #29968
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-01 18:53:03 +08:00
yihao.dai
a434d33e75
feat: Add import scheduler and manager ( #29367 )
...
This PR introduces novel managerial roles for importv2:
1. ImportMeta: To manage all the import tasks;
2. ImportScheduler: To process tasks and modify their states;
3. ImportChecker: To ascertain the completion of all tasks and instigate
relevant operations.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-01 18:31:02 +08:00
groot
85de56e894
fix: Clean kafka default configuration ( #30924 )
...
issue: #30917
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2024-03-01 18:17:03 +08:00
Bingyi Sun
7783098ddd
feat: support lazy load on querycoord ( #30372 )
...
https://github.com/milvus-io/milvus/issues/30361
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-01 18:15:29 +08:00
wayblink
f3c56c83ab
fix: Fix binlog_io metric name conflict ( #30689 )
...
follow: #29725
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-03-01 18:13:02 +08:00
zhagnlu
8b25ecffff
fix: add pk sort for same distance of hybridsearch ( #30901 )
...
#29923
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-03-01 18:09:02 +08:00
cai.zhang
41f1e6dcad
enhance:Add integration test for searching when coord down ( #30717 )
...
issue: #29772
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-01 17:07:00 +08:00
cai.zhang
1aa97a5c21
enhance: Support more relational operators for binary expressions ( #30902 )
...
issue: #30677
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-01 16:57:00 +08:00
SimFG
ee8d6f236c
enhance: make the watch dm channel request better compatibility ( #30952 )
...
issue: #30938
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-03-01 16:07:37 +08:00
chyezh
dd957cf9e3
enhance: add configurable memory index load predict memory usage factor ( #30561 )
...
related pr: https://github.com/milvus-io/milvus/pull/30475
Signed-off-by: chyezh <chyezh@outlook.com>
2024-03-01 15:23:00 +08:00
wei liu
cc46d6bafc
fix: segment version doesn't update as expected ( #30951 )
...
issue: #30950
due to segment version doesn't update as expected.
This PR will update segment version until segment become loaded
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-01 14:21:10 +08:00
congqixia
6387403639
fix: Prevent clone when selecting segments from meta ( #30928 )
...
See also #30538
Previously the `SelectSegments` changed to clone all return value
preventing possible update to returned info.
Since meta is implemented following COW rules, this shall not happen and
any update on segment shall have copy before it.
This PR:
- Remove clone for read-only Get segment info
- Add Segment Operator abstraction for changing segment
- Implemnt COW for updating MaxRowNum
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-01 13:59:02 +08:00
congqixia
385dec3b69
fix: index attr caches wrong result variable ( #30960 )
...
See also #30757 #30756
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-01 13:17:00 +08:00
aoiasd
2180e2cfae
fix: wrong segment binlog path cause load segment failed #30726 ( #30959 )
...
DataCoord GetSegmentInfo should return binlog info with logpath instead
logid when segment merge child segment's
binlog.
relate: https://github.com/milvus-io/milvus/issues/30366
https://github.com/milvus-io/milvus/issues/30165
https://github.com/milvus-io/milvus/issues/30550
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-01 13:09:01 +08:00
aoiasd
6f184d354f
enhance: Refine describe collection error message when collection not found ( #30766 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-01 12:53:00 +08:00
XuanYang-cn
2867f50fcc
fix: Clear DN unkown compaction tasks ( #30850 )
...
If DC restarted, those unkonwn compaction tasks
will never get call back in DN, so that the segments in the compaction
task will be locked, unable to sync and compaction again, blocking cp
advance and compaction executing.
See also: #30137
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-03-01 11:31:00 +08:00
cai.zhang
72ac2b1e93
fix:Update tencent cloud version ( #30847 )
...
Update tencent cloud version for updating token when expired.
issue: #30846
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-01 11:09:00 +08:00
PowderLi
4869aaeb94
fix: [restful v2] align search api with milvus client ( #30946 )
...
issue: #30688
fix: [restful v2] align search api with milvus client #30688
fix: [restful v2] DescribeIndex return Response{ Status: IndexNotFound
}, nil #30722
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-03-01 11:01:01 +08:00
zhagnlu
21b41e96fc
fix: Add hybridsearch result cases ( #30834 )
...
#30694
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-03-01 10:59:01 +08:00
Bingyi Sun
8addd75481
fix: Mmap could be enabled on ScaNN index ( #30914 )
...
issue: https://github.com/milvus-io/milvus/issues/30899
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-01 10:56:59 +08:00
aoiasd
3633923bb7
enhance: clean invalid pipline excluded segment info ( #30429 )
...
relate: https://github.com/milvus-io/milvus/issues/30281
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-01 10:43:01 +08:00
aoiasd
f3d1c75499
enhance: optimize access log formatter and add bench test ( #30829 )
...
relate: https://github.com/milvus-io/milvus/issues/28948
https://github.com/milvus-io/milvus/issues/30806
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-01 10:41:00 +08:00
congqixia
36d78e3dd0
fix: Use localStorage path to check disk cap ( #30944 )
...
See also #30943
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-01 10:17:01 +08:00
yihao.dai
a5a4ca8459
enhance: Remove debug log ( #30955 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-01 10:02:59 +08:00
MrPresent-Han
17a2fd048e
feat: support set up knowhere-build-pool-size on querynode( #29650 ) ( #30922 )
...
related: #29650
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-02-29 18:15:00 +08:00
yanliang567
d930666b3e
test: Add a test for issue #30607 ( #30933 )
...
related issue: #30607
and update some test for groupby
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-02-29 18:05:00 +08:00
chyezh
0c7474d7e8
enhance: add graceful stop timeout to avoid node stop hang under extreme cases ( #30317 )
...
1. add coordinator graceful stop timeout to 5s
2. change the order of datacoord component while stop
3. change querynode grace stop timeout to 900s, and we should
potentially change this to 600s when graceful stop is smooth
issue: #30310
also see pr: #30306
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-29 17:01:50 +08:00
Bingyi Sun
816ed671aa
fix: alter_index should return error if index not found ( #30786 )
...
issue: https://github.com/milvus-io/milvus/issues/30932
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-02-29 15:55:01 +08:00
Jiquan Long
dcfc35316f
fix: underscore matching not work ( #30826 )
...
issue: https://github.com/milvus-io/milvus/issues/30721
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-02-29 10:41:00 +08:00
zhuwenxing
b023bd09b3
test: skip bulk insert test for one pod standalone ( #30780 )
...
skip bulk insert test for one pod standalone
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-02-29 10:31:08 +08:00
chyezh
e59bc1e526
enhance: use latest milvus proto, add unimplemented server ( #30889 )
...
issue: #30647
Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-29 10:13:00 +08:00
PowderLi
50a78b682e
fix: set proxy.http.acceptTypeAllowInt64: true as default ( #30720 )
...
issue: #30680
also let the parameter item to be refreshable
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-02-29 09:59:07 +08:00
yihao.dai
3775464b7c
enhance: Support varchar autoid for bulkinsertV1 ( #30896 )
...
This PR is a supplement to PR
https://github.com/milvus-io/milvus/pull/30377 .
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-02-28 21:20:59 +08:00
sre-ci-robot
095cdbed59
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-02-28 12:01:10 +00:00
XuanYang-cn
cdc5ce5d6f
fix: Donot set LogPath when executing compaction ( #30537 )
...
Compaction would copy logPaths from comapctFrom segA to compactTo segB,
and previous code would copy the logPath directly, causing there're
full-logPaths-of-segA in compactTo segB's meta. So, for the next
compaction of segB, if segA has been GCed, Download would report error
"The sperified key not found".
This PR makes sure compactTo segment's meta contains logID only. And
this PR also refines CompleteComapctionMutation, increasing some
readability and merge two methods into one.
See also: #30496
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-02-28 19:03:01 +08:00
congqixia
4082315bd0
enhance: Add ParseCTraceContext util function for tracing ( #30883 )
...
See also #29803
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-02-28 18:59:00 +08:00
aoiasd
81a2d9ced6
enhance: access log support get sdk type by user agent ( #30760 )
...
Support get sdk type by user agent when we can't get sdk version by
connection in access log.
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-02-28 18:47:00 +08:00
groot
ba6d33cd57
fix: Support TLS for kafka connection ( #30468 )
...
#27977
Add extra configurations in milvus.yaml to pass certificates for kafka.
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2024-02-28 18:43:07 +08:00
Jiquan Long
e2f35954d4
enhance: support pattern matching on json field ( #30779 )
...
issue: https://github.com/milvus-io/milvus/issues/30714
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-02-28 18:31:00 +08:00
sammy.huang
ed1197ea50
enhance: Revert[skip e2e] bump milvus version for scripts/standalone_embed.sh ( #30892 )
...
This reverts commit cdb3f61fc8
2024-02-28 16:55:06 +08:00
