Commit Graph

408 Commits

Author SHA1 Message Date
yah01
bf633bb5d7
enhance: refine the retry error (#28573)
return the last error but not combining all errors, to improve
readability and erorr handling

resolve: #28572

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-30 18:34:32 +08:00
shaoyue
8b2b0d412c
enhance: storeageType default value change to remote (#28792)
/kind enhancement
/cc @PowderLi

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2023-11-30 11:34:27 +08:00
congqixia
f9bb8e9648
enhance: Change const magic number in querycoord to param (#28819)
See also #28817

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-30 09:06:28 +08:00
cqy123456
3b1b14dd78
fix: update binlog index memory uasge before loading segments (#28528)
issue: #27678 
when interimIndex = true, memory predict should be update with the
memory usage of binlog index build process.

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-11-29 16:42:27 +08:00
Gao
5bd73d5503
enhance: add efficient distance computations in Go (#28657)
Related #28656
Add more efficient calc_distance at Go side.

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-11-28 18:20:26 +08:00
XuanYang-cn
606ec77b66
enhance: Unify levelzero segment config in DN (#28720)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-28 18:04:26 +08:00
aoiasd
89d8ce2f73
enhance: refine access log to support format access log by yaml and print name info. (#28319)
relate: https://github.com/milvus-io/milvus/issues/28086

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-28 15:32:31 +08:00
cai.zhang
1b7a503f89
enhance: Revert import support csv format (#28760)
Revert import support csv format.
issue: #28778

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-28 14:32:27 +08:00
aoiasd
ce2436127c
fix: log can not reduce level (#28750)
relate: https://github.com/milvus-io/milvus/issues/28749

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-28 10:02:27 +08:00
congqixia
dd682658da
enhance: Add channel checkpoint ts lag metric (#28767)
See also #28765

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-28 07:00:28 +08:00
SimFG
9c46788d87
enhance: Support to trace restful request and request error (#28685)
issue: #28348

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-27 20:14:26 +08:00
cai.zhang
c29b60e18e
enhance: Support Array DataType for bulk_insert (#28341)
issue: #28272 
Support array DataType for bulk_insert with json, binlog files.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-27 13:50:27 +08:00
aoiasd
8a4cfb7d6a
enhance: add l0 metric and fix datacoord no need drop l0 segment when flush (#28373)
relate: https://github.com/milvus-io/milvus/issues/27675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-24 15:58:24 +08:00
Bingyi Sun
2843664a20
fix: channel name can not contains '-partition-' (#28689)
issue: https://github.com/milvus-io/milvus/issues/28675

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-11-24 10:14:24 +08:00
wei liu
4b8d3eaa41
enhance: Increase heartbeat timeout (#28648)
it's easy to trigger heartbeat timeout after 100ms when standalone cpu
usage reach 100%.
This PR increase the heartbeat timeout param to 2000ms

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-23 19:50:24 +08:00
congqixia
a2fe9dad49
enhance: Make etcd kv request timeout configurable (#28661)
See also #28660
This pr add request timeout config item for etcd kv request timeout
 Sync the default timeout value to same value for etcdKV & tikv config

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-23 19:34:23 +08:00
zhagnlu
bc43c5cf34
fix: Panic if minio port is empty (#28461)
#28460

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-11-23 19:24:25 +08:00
Bingyi Sun
4fedff6d47
feat: integrate storage v2 into the write path (#28440)
#28378

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-11-23 17:26:24 +08:00
SimFG
de13865769
enhance: Add load/release partitions to replicate msg stream (#28399)
/kind improvement
issue: #25655

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-23 15:38:24 +08:00
yah01
c96d07682e
enhance: enhance the accuracy of memory usage (#28554)
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.

we allocates the memory by private anon mmap,
then `rss - shared` would be the accurate memory usage

resolve #28553

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-23 15:12:23 +08:00
Enwei Jiao
3c3a57cb92
fix: Fix kafka config type error (#28642)
issue https://github.com/milvus-io/milvus/issues/28588

---------

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-11-23 14:36:22 +08:00
yah01
bfccfcd0ca
enhance: refine error messages (#28424)
- Split the simple reason and full detail
- Refine existing error messages
related: #28422

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-21 17:02:24 +08:00
XuanYang-cn
e88bbaac24
feat: Add universal levelzero segment switch (#28483)
See also: #27349

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-20 19:04:22 +08:00
PowderLi
7029797452
Improve HTTP server include metrics and RESTful API (#28045)
/kind improvement
issue: #27653

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-19 17:14:20 +08:00
congqixia
f1fc19e8a9
enhance: Add unittest for config.EventDispatcher (#28552)
These is no enough unittest case for EventDispatcher, see also #28540. 
This PR add unit test case for all methods for EventDispatcher
Related to #28538

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-18 19:12:20 +08:00
congqixia
c948a437a9
fix: protect EventDispatcher map with mutex (#28540)
Add mutex protection for `EventDispatcher.registry` map 
Fix #28538

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-17 20:32:20 +08:00
yah01
e9ff7ed13d
enhance: remove legacy error related code (#28385)
/kind improvement

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-14 15:02:18 +08:00
congqixia
371875f650
Fix MQ Factory timeout unit test (#28395)
This PR change timeout case to a longer duration in case of slow runner
Also change timeout case behavior of using timeout deadline instead of waiting
See also #28394

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-13 21:44:18 +08:00
SimFG
cfb6edea61
Support to trace the grpc request (#28349)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-13 20:16:18 +08:00
XuanYang-cn
f8aa46419a
Add LevelZeroCompaction configs (#28190)
See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-13 11:18:19 +08:00
yah01
6f59510723
Upgrade go to 1.20 (#24390)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-11-10 07:46:18 +08:00
SimFG
7dda2e8814
Change some log level in the pkg package (#28181)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-08 23:34:22 +08:00
zhenshan.cao
2057fdce43
Change default storage type to minio (#28236)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-11-07 21:15:31 +08:00
wei liu
5b45a138b1
disable auto balance when old node exists (#28191)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-07 14:02:20 +08:00
Xiaofan
da19e49daf
Support purge old session for standalone (#28184)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-11-06 21:21:42 +08:00
yah01
90e2c63d9e
Fix getting incorrect CPU num (#28146)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-11-06 06:02:16 +08:00
congqixia
bf2f62c1e7
Add WriteBuffer to provide abstraction for delta policy (#27874)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-04 12:10:17 +08:00
SimFG
ef68680639
Fix the unstable resource manager testcase (#28033)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-03 11:00:17 +08:00
aoiasd
d2727cc0ff
Add write cache for access log (#27792)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-03 10:28:17 +08:00
yah01
dc89730a50
Support collection-level mmap control (#26901)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-02 23:52:16 +08:00
wei liu
3b8fbbd91d
avoid sleep to ctx done during retry (#27992)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-01 18:52:24 +08:00
PowderLi
0c0f012e03
add a custom http header: Accept-Type-Allow-Int64 (#27901)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-01 11:42:16 +08:00
Enwei Jiao
8ae9c947ae
Use OpenDAL to access object store (#25642)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-11-01 09:00:14 +08:00
cqy123456
4fbe3c9142
replace loaded binlog with binlog index for search performance (#27673)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-11-01 02:20:15 +08:00
yah01
9658367a3c
Refine chunk manager errors (#27590)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-31 12:18:15 +08:00
yah01
2af46d7333
Increase the ChunkManager request timeout (#28015)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-31 09:06:13 +08:00
MrPresent-Han
ccaec44930
enable parameter for dataCoord broker timeout(#26972) (#27825)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-10-28 06:14:11 +08:00
yihao.dai
f9c630247d
Construct plan directly when search with vector output (#27928)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-26 19:30:10 +08:00
congqixia
8ab01e5a14
Set ElementType for empty array FieldData (#27924)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-25 20:08:10 +08:00
Gao
c8415453a1
Support prepare config for index (#27920)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-10-25 19:54:12 +08:00