wei liu
fceff6ed7f
enhance: Optimize param cost in search ( #37738 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-19 10:14:40 +08:00
congqixia
6d86b9022e
enhance: Provide secondary index critria when filter leaderview ( #37777 )
...
Related to #37630
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-19 10:12:30 +08:00
smellthemoon
3d28d99411
fix: to use the correct offset in span ( #37780 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 12 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
#37734
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-18 21:56:30 +08:00
congqixia
97b7ebbc08
fix: Move init kafka pool into once ( #37786 )
...
Introduced by #37744
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-18 21:24:48 +08:00
wei liu
351463b67e
fix: L0 segment has been loaded to worker during channel balance ( #37748 )
...
issue: #37703
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-18 20:58:30 +08:00
MimoN
5065d778d6
fix: [skip e2e]update webui base api url ( #37779 )
...
issue: #36621
- fix tools links in webui
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-11-18 20:40:31 +08:00
sthuang
2d72ad33f2
enhance: RBAC built in privilege groups ( #37720 )
...
issue: #37031
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-18 20:38:39 +08:00
aoiasd
16e206167c
enhance: analyzer length filter max should be close interval instead open interval ( #37770 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-18 19:30:31 +08:00
MimoN
82d2ac6be0
fix: update webui base api url ( #37775 )
...
issue: #36621
- fixed the base api url in webui
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-11-18 19:10:33 +08:00
wei liu
261212ee4a
fix: Get shard client failed by client is closed ( #37729 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 12 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
Weekly Release / Run Weekly Release (push) Has been cancelled
issue: #37718
This PR refine the shard client ref counter, dec ref counter won't
release client anymore, and only permit shard client manager to remove
client.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-18 18:22:30 +08:00
wei liu
226fe900e7
enhance: Use batch to speed up list collections from meta kv ( #37742 )
...
issue: #36228
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-18 16:58:30 +08:00
yihao.dai
0fc0d1a888
fix: Limit the concurrency of channel tasks ( #37740 )
...
Limit the maximum concurrency of channel tasks for each DataNode to
prevent excessive subscriptions from causing DataNode OOM.
issue: https://github.com/milvus-io/milvus/issues/37665
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-18 16:26:30 +08:00
aoiasd
e9391acf80
fix: bm25 brute force search need index params k1 and b ( #37721 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-18 15:44:31 +08:00
jaime
12ed40e125
fix: some missing metrics for WebUI restful API ( #37747 )
...
issue: #36621
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-18 14:00:32 +08:00
Chun Han
f2a2fd6808
enhance: shallow copy search request for large nq cases( #37732 ) ( #37749 )
...
related: #37732
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-11-18 13:06:32 +08:00
jaime
257ecab84b
enhance: remove collection queryable check from health check ( #37712 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-18 10:50:38 +08:00
congqixia
b0bd290a6e
enhance: Use internal json(sonic) to replace std json lib ( #37708 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 12 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
Related to #35020
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-18 10:46:31 +08:00
congqixia
3743fc2b00
enhance: Use pool to limit kafka cgo thread number ( #37744 )
...
Related to #37665
Thread number went rocket high when there is lots of kafka consumers on
datanode. Since the internal implementation is CGO, using which directly
will make cgo thread leaked.
This PR add a worker pool for kafka API utilzing CGO calls to limit
thread number.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-18 10:42:31 +08:00
Zhen Ye
3f1614e9d9
enhance: add trace_id into segcore logs ( #37656 )
...
issue: #37655
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-18 10:20:30 +08:00
Buqian Zheng
00edec2ebd
fix: incorrect dim set when creating BM25 doc sparse array ( #37717 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Has been cancelled
Code Checker / Code Checker Amazonlinux 2023 (push) Has been cancelled
Code Checker / Code Checker rockylinux8 (push) Has been cancelled
Mac Code Checker / Code Checker MacOS 12 (push) Has been cancelled
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Has been cancelled
Build and test / UT for Cpp (push) Has been cancelled
Build and test / UT for Go (push) Has been cancelled
Build and test / Integration Test (push) Has been cancelled
Build and test / Upload Code Coverage (push) Has been cancelled
SIMD Compatibility Test / test-simd-compatibility (avx) (push) Has been cancelled
SIMD Compatibility Test / test-simd-compatibility (avx2) (push) Has been cancelled
SIMD Compatibility Test / test-simd-compatibility (avx512) (push) Has been cancelled
SIMD Compatibility Test / test-simd-compatibility (sse4_2) (push) Has been cancelled
issue: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-11-16 16:36:30 +08:00
zhuwenxing
3cdb485022
test: update timeout of load in ci cases ( #37596 )
...
action for
https://github.com/milvus-io/milvus/issues/37166#issuecomment-2469502955
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-16 16:00:31 +08:00
congqixia
4820dd76e5
fix: Close leaked health check kafka producer ( #37728 )
...
Related to #36822
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-16 11:20:32 +08:00
zhuwenxing
3f7352f3cf
test: remove xfail of fts test cases after fix ( #37724 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-16 11:14:30 +08:00
jaime
ead1e7f68c
enhance: The Restful API supports a single page web application. ( #37719 )
...
issue: #36621
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-16 11:04:31 +08:00
yihao.dai
3df2c92bb0
enhance: Remove segment-level tag from monitoring metrics ( #37696 )
...
When there are a large number of segments, the metrics consume a lot of
memory. This PR Remove segment-level tag from monitoring metrics.
issue: https://github.com/milvus-io/milvus/issues/37636
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-16 10:58:31 +08:00
MimoN
f7c7ac51fc
fix: update webui and remove the unnecessary code ( #37726 )
...
issue: #36621
* update webui top nav
* remove the unnecessary code in .gitignore file
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-11-16 10:56:33 +08:00
aoiasd
3b5a0df159
enhance: Optimize chinese analyzer and support CnAlphaNumFilter ( #37727 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-16 10:28:30 +08:00
foxspy
0ba868ae64
enhance: update knowhere version ( #37730 )
...
release note draft :
https://github.com/zilliztech/knowhere/releases/tag/v2.5.0
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-11-16 10:08:30 +08:00
sre-ci-robot
589b5b9786
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-11-15 12:00:46 +00:00
smellthemoon
7999367c0c
fix: use not retried err when get wrong parameter ( #37707 )
...
#37508
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-15 19:14:30 +08:00
zhagnlu
e4b6773d0a
fix: fix create text index dir conflict bug ( #37693 )
...
#37623
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-11-15 18:26:30 +08:00
MimoN
0f0162f70a
feat: refactor web ui ( #37698 )
...
issue: #36621
* The front-end code of the management interface has been refactored
using React.
* The interaction and display of the front-end interface have been
optimized.
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
Co-authored-by: zilliz <zilliz@zillizdeMacBook-Pro.local>
2024-11-15 17:20:30 +08:00
yihao.dai
18a64782dc
enhance: Remove unnecessary clone in SetState ( #37697 )
...
issue: https://github.com/milvus-io/milvus/issues/37637
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-15 15:06:30 +08:00
Zhen Ye
81fa7dd52c
fix: add ddl and dcl concurrency to avoid competition ( #37672 )
...
issue: #37166
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-15 15:04:31 +08:00
Bingyi Sun
65d3c6622a
enhance: Optimize GetChunkIDByOffset and add ut ( #37704 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-15 14:16:31 +08:00
XuanYang-cn
5a23c80f20
fix: Change memoryCheck write lock to read lock ( #37525 )
...
See also: milvus-io#37493
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-15 10:44:31 +08:00
Bingyi Sun
d1596297d9
fix: Fix query failure with inverted index ( #37686 )
...
https://github.com/milvus-io/milvus/issues/37649
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-15 10:28:31 +08:00
congqixia
5afcee6bfa
fix: Store default value if ErrKeyNotFound is returned ( #37691 )
...
Related to #37690
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-15 10:24:32 +08:00
congqixia
66bf254437
enhance: Enable RemoteLoad l0 forward policy by default ( #37678 )
...
Related to #35303
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-15 10:22:30 +08:00
wei liu
a1b6be1253
fix: Delegator stuck at unserviceable status ( #37694 )
...
issue: #37679
pr #36549 introduce the logic error which update current target when
only parts of channel is ready.
This PR fix the logic error and let dist handler keep pull distribution
on querynode until all delegator becomes serviceable.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-15 10:20:31 +08:00
cai.zhang
b9357e4716
fix: Modify the batchsize of writer to timely flushing binlogs ( #37692 )
...
issue: #37579
If the schema includes large varchar fields, a few thousand rows can
reach hundreds of MB in size. Therefore, if the batch size of the
segment writer is large, it will produce relatively large `binlogs`,
which can cause datanode to run out of memory (OOM) during compaction.
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-15 10:14:31 +08:00
foxspy
5ae347aba0
enhance: update knowhere version ( #37688 )
...
issue: #37665 #37631 #37620 #37587 #36906
knowhere has add default nlist value, so some invalid param test ut with
no nlist param will be valid.
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-11-15 10:10:31 +08:00
jaime
1d06d4324b
fix: Int64 overflow in JSON encoding ( #37657 )
...
issue: ##36621
- For simple types in a struct, add "string" to the JSON tag for
automatic string conversion during JSON encoding.
- For complex types in a struct, replace "int64" with "string."
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-14 22:52:30 +08:00
wei liu
c5485bb1b1
fix: Make GetShardLeaders skip retry on not loaded collection ( #37684 )
...
issue: #37532
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-14 21:14:40 +08:00
Bingyi Sun
1b4f7e3ac1
enhance: Add more expr ut for chunked segment ( #37600 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Has been cancelled
Code Checker / Code Checker Amazonlinux 2023 (push) Has been cancelled
Code Checker / Code Checker rockylinux8 (push) Has been cancelled
Mac Code Checker / Code Checker MacOS 12 (push) Has been cancelled
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Has been cancelled
Publish Test Images / PyTest (push) Has been cancelled
Build and test / UT for Cpp (push) Has been cancelled
Build and test / UT for Go (push) Has been cancelled
Build and test / Integration Test (push) Has been cancelled
Build and test / Upload Code Coverage (push) Has been cancelled
Weekly Release / Run Weekly Release (push) Has been cancelled
related pr: #37570
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-14 18:40:32 +08:00
nico
405c12f7d1
test: update test cases ( #37660 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-11-14 17:46:39 +08:00
aoiasd
993051bb49
fix: brute force bm25 search lack avgdl param ( #37650 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-14 14:58:31 +08:00
XuanYang-cn
31a8d08bdd
fix: Correct varchar primarykey size calculation ( #37617 )
...
See also: #37582
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-14 14:16:38 +08:00
Zhen Ye
cd181e4c6d
fix: make NewDataSyncService idempotent of dispatcher ( #37576 )
...
issue: #37547
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-14 11:00:31 +08:00
Buqian Zheng
0565300b7f
fix: Sparse to use CC index as growing/temp index ( #37591 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-11-14 10:54:31 +08:00
