test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-11-29 18:38:44 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
20,756 Commits 94 Branches 105 Tags 184 MiB
763fd0dfc5
Commit Graph

599 Commits

Author SHA1 Message Date
yihao.dai
763fd0dfc5
enhance: Use a separate mmap config for chunk cache (#36276) 
issue: https://github.com/milvus-io/milvus/issues/35273

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-09-15 16:23:09 +08:00
Chun Han
b8b4aea4f5
enhance: restrict max group size(#33544) (#36223) 
related: #33544

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-09-14 15:45:08 +08:00
aoiasd
c22a2cebb6
fix: split stream query result to avoid grpc response too large error (#36090) 
relate: https://github.com/milvus-io/milvus/issues/36089

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-09-13 15:07:09 +08:00
zhagnlu
5e5e87cc2f
enhance: rename some params and reduce default bitmapCardinalityLimit… (#36138) 
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-09-12 12:09:08 +08:00
congqixia
11dbe1e755
enhance: Add L0 forward policy to support remote load (#36189) 
Related to #35303

This PR add a param item to support change l0 forward behavior from bf
filtering and forward to remote load.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-12 12:01:08 +08:00
XuanYang-cn
f31264c02c
fix: Change l0SegmentsRowCount limits to a reasonable value (#36014) 
See also: #36028

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-09-11 11:55:08 +08:00
zhagnlu
208c8a2328
fix:support config index offsetcache and fix create same index again (#35985) 
#35971

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-09-08 18:23:05 +08:00
Ted Xu
d9a40784a2
fix: fallback params may be overridden (#35972) 
See #35756

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-09-05 16:19:04 +08:00
congqixia
8593c4580a
enhance: Add delete buffer related quota logic (#35918) 
See also #35303

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-05 11:39:03 +08:00
jaime
24fb10114b
enhance: remove cooling off in rate limiter for read requests (#35935) 
issue: #35934

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-09-04 14:39:10 +08:00
Zhen Ye
99dff06391
enhance: using streaming service in insert/upsert/flush/delete/querynode (#35406) 
issue: #33285

- using streaming service in insert/upsert/flush/delete/querynode
- fixup flusher bugs and refactor the flush operation
- enable streaming service for dml and ddl
- pass the e2e when enabling streaming service
- pass the integration tst when enabling streaming service

---------

Signed-off-by: chyezh <chyezh@outlook.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-29 10:03:08 +08:00
SimFG
731d45abbe
enhance: provide more general configuration to control mmap behavior (#35359) 
- issue: #35273

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-21 00:22:54 +08:00
Ted Xu
41646c8439
feat: integrate new deltalog format (#35522) 
See #34123

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-20 19:06:56 +08:00
Ted Xu
c6ae7d4d56
enhance: adding the msgchannel section in generated yaml (#35466) 
See #32168

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-14 18:32:53 +08:00
presburger
a9352a073b
enhance: change gpu default mem pool size (#34159) 
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
 2024-08-06 10:24:15 +08:00
cqy123456
b99a554827
enhance:update mmap_chunk_manager config in milvus.yaml (#35180) 
issue: https://github.com/milvus-io/milvus/issues/32984
related pr: https://github.com/milvus-io/milvus/pull/35186

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-08-02 19:28:15 +08:00
jaime
fcec4c21b9
fix: check collection health(queryable) fail for releasing collection (#34947) 
issue: #34946

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-08-02 17:20:15 +08:00
congqixia
c64a078458
enhance: Support proxy/delegator qn client pooling (#35194) 
See also #35196

Add param item for proxy/delegator query node client pooling and
implement pooling logic

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 11:24:19 +08:00
cai.zhang
196a7986b3
enhance: Change the fixed value to a ratio for clustering segment size (#35076) 
issue: #34495

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-01 22:04:14 +08:00
Ted Xu
b4d0f4df0a
enhance: update param documents (#35152) 
See: #35161

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-01 12:38:13 +08:00
wei liu
e9d61daa3f
enhance: Reduce delegator memory overloaded factor to 0.1 (#35092) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-01 10:21:50 +08:00
wayblink
ce3f836876
fix: compaction task not be cleaned correctly (#34765) 
1.fix compaction task not be cleaned correctly
2.add a new parameter to control compaction gc loop interval
3.remove some useless configs of clustering compaction

bug: #34764

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-30 20:21:56 +08:00
congqixia
de8a266d8a
enhance: Enable linux code checker (#35084) 
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 15:53:51 +08:00
congqixia
972752258a
enhance: Support otlp http exporter (#35053) 
See also #35052

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-29 17:43:49 +08:00
Ted Xu
63f0154dfb
fix: enable milvus.yaml check (#34567) 
See #32168

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-07-25 18:05:46 +08:00
smellthemoon
5616b7e8d2
enhance: support null in c data_datacodec and load null value (#32183) 
1. support read and write null in segcore
    will store valid_data(use uint8_t type to save memory) in fieldData.
2. support load null
binlog reader read and write data into column(sealed segment),
insertRecord(growing segment). In sealed segment, store valid_data
directly. In growing segment, considering prior implementation and easy
code reading, it covert uint8_t to fbvector<bool>, which may optimize in
future.
3.  retrieve valid_data.
    parse valid_data in search/query.
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-07-23 16:07:51 +08:00
wayblink
c2b8b5fe84
enhance: refine clustering compaction configs and logs (#34784) 
#30633

---------

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-21 19:23:40 +08:00
yihao.dai
b22e549844
enhance: Rename config of sealing by growing segmetns size (#34787) 
/kind improvement

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-19 20:27:41 +08:00
chyezh
86eff6e589
enhance: streaming node client implementation (#34653) 
issue: #33285

- add streaming node grpc client wrapper
- add unittest for streaming node grpc client side
- fix binary unsafe bug for message

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-07-19 17:37:40 +08:00
wayblink
c79d1af390
enhance: Add compaction task slot usage logic (#34581) 
#34544

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-18 10:27:41 +08:00
yihao.dai
4939f82d4f
enhance: Seal by total growing segments size (#34692) 
Seals the largest growing segment if the total size of growing segments
of each shard exceeds the size threshold(default 4GB). Introducing this
policy can help keep the size of growing segments within a suitable
level, alleviating the pressure on the delegator.

issue: https://github.com/milvus-io/milvus/issues/34554

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-17 21:45:41 +08:00
SimFG
203fb554a4
enhance: support to config root user's password (#34752) 
- issue: #33058

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-17 20:19:42 +08:00
congqixia
8b5754f7fe
enhance: Add segment seal proportion jitter (#34636) 
See also #34574

Add jitter for segment seal proportion to avoid seal operation burst in
short period of time.

This PR also fix license header in paramtable pkg.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-15 14:47:39 +08:00
congqixia
1a248f2668
enhance: Add param item for segmentFlushInterval (#34629) 
See also #28817

Add paramitem for segment flush interval

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-12 18:59:35 +08:00
aoiasd
c1e04534c3
enhance:change access log write cache default config (#34354) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-07-09 18:18:12 +08:00
jaime
21fc5f5d46
enhance: Remove datanode reporting TT based on MQ implementation (#34421) 
issue: #34420

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-05 15:48:09 +08:00
XuanYang-cn
8a2be8a457
fix: DataNode might OOM by estimating based on MemorySize (#34201) 
See also: #34136

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-04 10:16:17 +08:00
jaime
d6afb31b94
enhance: make subfunctions of datanode component modular (#33992) 
issue: #33994

also remove deprecated channel manager based on the etcd implementation

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-01 14:46:07 +08:00
XuanYang-cn
dda70aa81b
fix: LegacyVersionWithoutRPCWatch default value to 2.4.1 (#34184) 
See also: #31933

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-27 19:36:06 +08:00
wayblink
f9a0f7bb25
Add an option to enable/disable vector field clustering key (#34097) 
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-06-25 18:52:04 +08:00
jaime
d08cb885ca
enhance: enable flush rate limiter of collection level (#33837) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-24 10:52:03 +08:00
congqixia
cc77363b66
enhance: Set maxPartitionNum default value to 1024 (#33949) 
See also #30059

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-20 09:11:59 +08:00
chyezh
2f6f964bc8
enhance: [skip e2e] modify gc configuration document (#33946) 
issue: #31740

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-19 09:57:59 +08:00
cqy123456
32f685ff12
enhance: growing segment support mmap (#32633) 
issue: https://github.com/milvus-io/milvus/issues/32984

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-18 14:42:00 +08:00
XuanYang-cn
1629833060
enhance: Add consts of MsgDispatcher to configs (#33679) 
See also: #33676

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-07 14:21:59 +08:00
SimFG
ecee7d90d4
enhance: try to speed up the loading of small collections (#33570) 
- issue: #33569

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-06-07 08:25:53 +08:00
cai.zhang
27cc9f2630
enhance: Support analyze data (#33651) 
issue: #30633

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>
 2024-06-06 17:37:51 +08:00
cai.zhang
77637180fa
enhance: Periodically synchronize segments to datanode watcher (#33420) 
issue: #32809

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-05-30 13:37:44 +08:00
yihao.dai
bbb69980ac
enhance: Replace 'off' with 'disable' (#33433) 
YAML will automatically parse "off" as a boolean variable. We should
avoid using "off" in the future.

issue: https://github.com/milvus-io/milvus/issues/32772

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-29 12:17:43 +08:00
Bingyi Sun
6b3e42f8d8
fix: fix wrong default local storage path (#33389) 
issue: https://github.com/milvus-io/milvus/issues/33427

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-05-28 16:49:43 +08:00