XuanYang-cn
5a23c80f20
fix: Change memoryCheck write lock to read lock ( #37525 )
...
See also: milvus-io#37493
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-15 10:44:31 +08:00
Bingyi Sun
d1596297d9
fix: Fix query failure with inverted index ( #37686 )
...
https://github.com/milvus-io/milvus/issues/37649
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-15 10:28:31 +08:00
congqixia
5afcee6bfa
fix: Store default value if ErrKeyNotFound is returned ( #37691 )
...
Related to #37690
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-15 10:24:32 +08:00
congqixia
66bf254437
enhance: Enable RemoteLoad l0 forward policy by default ( #37678 )
...
Related to #35303
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-15 10:22:30 +08:00
wei liu
a1b6be1253
fix: Delegator stuck at unserviceable status ( #37694 )
...
issue: #37679
pr #36549 introduce the logic error which update current target when
only parts of channel is ready.
This PR fix the logic error and let dist handler keep pull distribution
on querynode until all delegator becomes serviceable.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-15 10:20:31 +08:00
cai.zhang
b9357e4716
fix: Modify the batchsize of writer to timely flushing binlogs ( #37692 )
...
issue: #37579
If the schema includes large varchar fields, a few thousand rows can
reach hundreds of MB in size. Therefore, if the batch size of the
segment writer is large, it will produce relatively large `binlogs`,
which can cause datanode to run out of memory (OOM) during compaction.
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-15 10:14:31 +08:00
foxspy
5ae347aba0
enhance: update knowhere version ( #37688 )
...
issue: #37665 #37631 #37620 #37587 #36906
knowhere has add default nlist value, so some invalid param test ut with
no nlist param will be valid.
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-11-15 10:10:31 +08:00
jaime
1d06d4324b
fix: Int64 overflow in JSON encoding ( #37657 )
...
issue: ##36621
- For simple types in a struct, add "string" to the JSON tag for
automatic string conversion during JSON encoding.
- For complex types in a struct, replace "int64" with "string."
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-14 22:52:30 +08:00
wei liu
c5485bb1b1
fix: Make GetShardLeaders skip retry on not loaded collection ( #37684 )
...
issue: #37532
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-14 21:14:40 +08:00
Bingyi Sun
1b4f7e3ac1
enhance: Add more expr ut for chunked segment ( #37600 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Has been cancelled
Code Checker / Code Checker Amazonlinux 2023 (push) Has been cancelled
Code Checker / Code Checker rockylinux8 (push) Has been cancelled
Mac Code Checker / Code Checker MacOS 12 (push) Has been cancelled
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Has been cancelled
Publish Test Images / PyTest (push) Has been cancelled
Build and test / UT for Cpp (push) Has been cancelled
Build and test / UT for Go (push) Has been cancelled
Build and test / Integration Test (push) Has been cancelled
Build and test / Upload Code Coverage (push) Has been cancelled
Weekly Release / Run Weekly Release (push) Has been cancelled
related pr: #37570
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-14 18:40:32 +08:00
nico
405c12f7d1
test: update test cases ( #37660 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-11-14 17:46:39 +08:00
aoiasd
993051bb49
fix: brute force bm25 search lack avgdl param ( #37650 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-14 14:58:31 +08:00
XuanYang-cn
31a8d08bdd
fix: Correct varchar primarykey size calculation ( #37617 )
...
See also: #37582
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-14 14:16:38 +08:00
Zhen Ye
cd181e4c6d
fix: make NewDataSyncService idempotent of dispatcher ( #37576 )
...
issue: #37547
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-14 11:00:31 +08:00
Buqian Zheng
0565300b7f
fix: Sparse to use CC index as growing/temp index ( #37591 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-11-14 10:54:31 +08:00
aoiasd
1c5b5e1e3d
feat: Add chinese and english analyzer with refactor jieba tokenizer ( #37494 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 12 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-14 10:34:31 +08:00
wei liu
1304b40552
fix: Balance channel may stuck at increasing replica number case ( #37641 )
...
issue: #37640
fix the pr #36549
cause balance channel will wait until new delegator becomes serviceable,
but new delegator need to sync target version then becomes serviceable,
and sync target version need to be wait all replica load done. so if
increasing replica number and balance channel happens at same time,
logic dead lock occurs.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-14 10:08:31 +08:00
sre-ci-robot
caf207f5a0
Update all contributors
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 12 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-11-13 12:00:38 +00:00
congqixia
e7fb87d491
enhance: Fix lb policy retry log after targetNode type changed ( #37646 )
...
Related to previous pr: #37371
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-13 19:52:29 +08:00
yanliang567
af433ffd68
test: Add expression template tests and use error msg only as assertion ( #37618 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 12 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
related issue: https://github.com/milvus-io/milvus/issues/37451
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-11-13 17:14:30 +08:00
smellthemoon
a654487995
fix: some error in restful ( #37487 )
...
#37370
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-13 17:12:39 +08:00
foxspy
cf883b114e
enhance: update knowhere version ( #37510 )
...
issue: #36925
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-11-13 16:36:27 +08:00
Ted Xu
a9538f6e96
fix: config file validator missed the file tail validation ( #37608 )
...
See: #32168
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-11-13 15:14:30 +08:00
Ted Xu
1a49da2cc0
fix: refuse schedule compaction tasks if there is no slot ( #37589 )
...
See #37621
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-11-13 15:12:30 +08:00
smellthemoon
3389a6b500
enhance: support null in text match index ( #37517 )
...
#37508
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-13 11:08:29 +08:00
sre-ci-robot
30d396f476
Update all contributors
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 12 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-11-13 00:01:25 +00:00
codingjaguar
cfdff464dd
doc: Rewrite readme file ( #37428 )
...
Signed-off-by: codingjaguar <jiang.chen@zilliz.com>
2024-11-12 13:29:57 -08:00
Zhen Ye
3c225e5c94
fix: data race when using fields_ ( #37612 )
...
issue: #37609
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-13 04:06:30 +08:00
XuanYang-cn
5ab3c562da
enhance: tidy compaction logs ( #37595 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 12 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
Remove some annoying logs and lower a log level from warn to info
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-12 21:18:29 +08:00
cai.zhang
14e007d6fb
fix: Fix the bug that retrieved from wrong field for L0 segments ( #37598 )
...
issue: #37574
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-12 20:58:29 +08:00
yellow-shine
cef5b7fd60
enhance: [cpp-unit-test] abort previous build if it's a PR, otherwise… ( #37571 )
...
[cpp-unit-test] abort previous build if it's a PR, otherwise queue the
build
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-11-12 19:14:29 +08:00
nico
2630717836
test: update sdk version and cases ( #37507 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 12 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
Publish Test Images / PyTest (push) Has been cancelled
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-11-12 17:48:28 +08:00
jaime
1e8ea4a7e7
feat: add segment/channel/task/slow query render ( #37561 )
...
issue: #36621
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-12 17:44:29 +08:00
Bingyi Sun
2742e9573f
enhance: Print results if a test fails ( #37606 )
...
test_hybrid_search_different_limit_round_decimal fails occasionally.
Print more details for debug.
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-12 17:12:28 +08:00
wei liu
266f8ef1f5
fix: Search may return less result after qn recover ( #36549 )
...
issue: #36293 #36242
after qn recover, delegator may be loaded in new node, after all segment
has been loaded, delegator becomes serviceable. but delegator's target
version hasn't been synced, and if search/query comes, delegator will
use wrong target version to filter out a empty segment list, which
caused empty search result.
This pr will block delegator's serviceable status until target version
is synced
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-12 16:34:28 +08:00
Zhen Ye
b5b003551e
enhance: use localhost for it and ut ( #37529 )
...
issue: #37528
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-12 11:36:27 +08:00
aoiasd
4682bba586
enhance: Support gc with bm25 stats logs ( #37408 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-12 11:30:28 +08:00
congqixia
24c6a4bb29
enhance: [GoSDK] support Doc-in-doc-out APIs ( #37590 )
...
Related to #35853
This PR contains following changes:
- Add function and related proto and helper functions
- Remove the insert column missing check and leave it to server
- Add text as search input data
- Add some unit tests for logic above
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-12 10:58:28 +08:00
XuanYang-cn
a45a288a25
fix: Separate L0 and Mix trigger interval ( #37190 )
...
See also: #37108
- Add MixCompactionTriggerInterval, default 60s
- Add L0CompactionTriggerInterval, default 10s
- Export Single related compaction configs
- Raise SingleCompactionDeltaLogMaxSize from 2MB to 16MB
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-12 10:56:37 +08:00
wei liu
2a4c00de9d
enhance: Decouple shard client manager from shard cache ( #37371 )
...
issue: #37115
the old implementation update shard cache and shard client manager at
same time, which causes lots of conor case due to concurrent issue
without lock.
This PR decouple shard client manager from shard cache, so only shard
cache will be updated if delegator changes. and make sure shard client
manager will always return the right client, and create a new client if
not exist. in case of client leak, shard client manager will purge
client in async for every 10 minutes.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-12 10:30:28 +08:00
congqixia
f5b06a3c9f
enhance: Invalidate collection cache when release collection ( #37577 )
...
Related to #37395
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-12 10:16:29 +08:00
Chun Han
2d29dcd30c
enhance:refine group_strict_size parameter( #37482 ) ( #37483 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 12 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
Publish Test Images / PyTest (push) Waiting to run
related: #37482
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-11-12 09:56:28 +08:00
binbin
21b68029a0
test: Add nullable test cases for bulk writer ( #37572 )
...
issue: #36129
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-11-12 09:46:28 +08:00
Bingyi Sun
c1eccce2fa
enhance: enable multiple chunked segment by default ( #37570 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-12 09:20:28 +08:00
foxspy
81141bd18d
fix: use yaml.v2 as yaml parser ( #37423 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 12 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
issue: #34298
Viper uses yaml.v2 as the parser. This PR will adopt the parsing logic
from Viper to handle YAML files, ensuring maximum consistency in
parsing.
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-11-11 21:26:27 +08:00
Zhen Ye
1b6edd0b4b
enhance: refactor the consumer grpc proto for reusing grpc stream for multi-consumer ( #37564 )
...
Code Checker / Code Checker AMD64 Ubuntu 22.04 (push) Waiting to run
Code Checker / Code Checker Amazonlinux 2023 (push) Waiting to run
Code Checker / Code Checker rockylinux8 (push) Waiting to run
Mac Code Checker / Code Checker MacOS 12 (push) Waiting to run
Build and test / Build and test AMD64 Ubuntu 22.04 (push) Waiting to run
Build and test / UT for Cpp (push) Blocked by required conditions
Build and test / UT for Go (push) Blocked by required conditions
Build and test / Integration Test (push) Blocked by required conditions
Build and test / Upload Code Coverage (push) Blocked by required conditions
Weekly Release / Run Weekly Release (push) Has been cancelled
issue: #33285
- Modify the proto of consumer of streaming service.
- Make VChannel as a required option for streaming
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-11 17:24:29 +08:00
XuanYang-cn
5e6c3df253
fix: l0RowCount metrics value always empty ( #37306 )
...
See also: #36953
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-11 17:00:27 +08:00
wei liu
61a5b15ada
fix: Lost loading collection's updateTs after qc restart ( #37538 )
...
issue: #37537
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-11 14:34:28 +08:00
congqixia
83d2729f94
enhance: Generate milvusclient mockery with latest proto ( #37567 )
...
Related to #37558
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-11 14:28:30 +08:00
Ted Xu
31d0c84f67
fix: panic calclulating data size on writing binlogs ( #37575 )
...
See: #37568
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-11-11 14:08:27 +08:00
