Commit Graph

18565 Commits

Author SHA1 Message Date
congqixia
0b905078e7
Use writebuffer, sync manager refactory in datanode (#28320)
See also #27675
This PR make previously merged refactory of datanode go online
- Use write node to replace insert/delete node
- Use write buffer manager to control all buffers
- Use sync manager to control sync tasks instead of flush manager

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-15 15:24:18 +08:00
yah01
fd3ae4c4b8
enhance: improve the compile speed (#28444)
improve the CI compile speed with mold

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-15 15:08:18 +08:00
SimFG
899a5a32cd
Hide the password info when failing to authorize (#28428)
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-15 14:40:26 +08:00
MrPresent-Han
836f300536
support skip-index based on chunk-metrics to accelerate expr filter(#27925) (#28297)
related: #27925

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-11-15 11:20:19 +08:00
Bingyi Sun
59355cb3dc
Update arrow version to v12 (#28425)
issue: https://github.com/milvus-io/milvus/issues/28423

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-11-15 10:36:19 +08:00
sre-ci-robot
6c66586935 Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2023-11-14 13:03:23 +00:00
XuanYang-cn
0be22ccacd
Fix compaction unable to notify datanode (#28409)
See also: #28214, #28368

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-14 16:38:18 +08:00
XuanYang-cn
40d5c902b6
Enable getting multiple segments in plan result (#28350)
Compaction plan result contained one segment for one plan. For l0
compaction would write to multiple segments, this PR expand the segments
number in plan results and refactor some names for readibility.

- Name refactory: - CompactionStateResult -> CompactionPlanResult -
CompactionResult -> CompactionSegment

See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-14 15:56:19 +08:00
XuanYang-cn
8037f35de7
Change MetaCache interface (#28388)
See also: #28320, #27675

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-14 15:08:19 +08:00
yah01
e9ff7ed13d
enhance: remove legacy error related code (#28385)
/kind improvement

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-14 15:02:18 +08:00
Xiaofan
2d7bcac848
Update README.md 2023-11-14 13:46:38 +08:00
ThreadDao
ec23107146
Add test cases for L0 delete (#28380)
- test delete records in delta logs, WAL, L0 segment

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2023-11-14 10:26:24 +08:00
congqixia
b1eb1ea506
Refine datanode Timetick Sender (#28393)
- Use explicit lifetime control methods: `Start` and `Stop`
- Allow control retry option
- Make sure tt sender worker exit after `Stop` return

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-13 21:46:20 +08:00
congqixia
371875f650
Fix MQ Factory timeout unit test (#28395)
This PR change timeout case to a longer duration in case of slow runner
Also change timeout case behavior of using timeout deadline instead of waiting
See also #28394

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-13 21:44:18 +08:00
SimFG
cfb6edea61
Support to trace the grpc request (#28349)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-13 20:16:18 +08:00
XuanYang-cn
87465f07a7
Fix unstable ConsistencyHashPolicy ut (#28375)
Fixes: #28372

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-13 19:06:20 +08:00
yihao.dai
c16fc854d1
Fix confusing quota log (#28262)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-11-13 15:38:18 +08:00
XuanYang-cn
f8aa46419a
Add LevelZeroCompaction configs (#28190)
See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-13 11:18:19 +08:00
XuanYang-cn
a153950b10
Change channel to Interface (#27839)
This PR changes `*channel` into RWChannel interface

See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-13 11:16:18 +08:00
smellthemoon
0aa90de141
Reduce the goroutine in flowgraph to 2 (#28233)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-13 10:50:17 +08:00
smellthemoon
5365748338
Use single instance for mergedTimeTickerSender (#27730)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-13 10:18:17 +08:00
nico
4d30405a6e
update test cases (#28339)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2023-11-13 09:32:31 +08:00
Xu Tong
8ec85f5f4c
Add template for VectorMemIndex (#28324)
Signed-off-by: Writer-X <1256866856@qq.com>
2023-11-11 13:20:22 +08:00
yah01
267c67dfee
enhance: reduce 1x copy while retrieving data from growing segment (#28323)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-10 15:44:22 +08:00
yah01
70995383bf
enhance: modify log to avoid ambiguity and improve readability (#28331)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-10 14:32:20 +08:00
zhagnlu
3920bbc55f
Force set aliyun use_virtual_host to true for all (#28158)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-11-10 10:32:20 +08:00
cai.zhang
01c5484817
No hit when the index exceeds the array length (#28301)
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-10 10:04:19 +08:00
yah01
6f59510723
Upgrade go to 1.20 (#24390)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-11-10 07:46:18 +08:00
wei liu
8a370f115a
Fix external grpc server doesn't graceful stop (#28309)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-10 00:14:18 +08:00
wei liu
14c8a90517
Fix auto balance block channel reassign after datanode restart (#28275)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-09 19:00:25 +08:00
wei liu
bce1054f92
Fix retry when proxy stopped (#28264)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-09 18:58:21 +08:00
yah01
30847cad3e
Handle exception while loading (#28304)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-09 17:59:12 +08:00
sre-ci-robot
b7b31ce0bc
Update knowhere commit (#28285)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-11-09 14:24:19 +08:00
congqixia
e576271a24
Fix buffer FieldData has no ElementType and array logsize always zero (#28295)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-09 14:16:20 +08:00
Enwei Jiao
cbf25f8585
Use OpenDAL 0.42 (#28258)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-11-09 11:26:21 +08:00
wei liu
b9bf910039
fix unstable auto balance config ut (#28288)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-09 10:00:22 +08:00
SimFG
7dda2e8814
Change some log level in the pkg package (#28181)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-08 23:34:22 +08:00
sre-ci-robot
cd294056fc
Update Pytest image changes (#28211)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-11-08 19:08:27 +08:00
wei liu
7f78e1dd46
fix datacoord unstable ut (#28281)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-08 18:43:31 +08:00
PowderLi
9f9726f8b7
print azure sdk log (#28240)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-08 17:50:18 +08:00
yah01
ecb3f585c3
Fix passing the wrong dropped list from current target (#28265)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-08 17:02:18 +08:00
yah01
d20ea061d6
Fix panic while forwarding empty deletions to growing segment (#28213)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-08 16:42:21 +08:00
jaime
ec0d9d8fd4
Fix delete session key with prefix (#28260)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-11-08 15:22:21 +08:00
yah01
f4341f254d
Not convert legacy error code to new merr (#28232)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-08 14:42:20 +08:00
yah01
1b90630633
Fix the target updated before version updated to cause data missing (#28250)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-08 11:36:22 +08:00
SimFG
e3b7fdac61
Delay the cancellation of ctx when stopping the node (#28247)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-08 03:20:17 +08:00
wei liu
64f6dbabdc
Fix proxy not healthy before graceful stop (#28223)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-08 02:22:26 +08:00
cai.zhang
2b5f632fa4
Fix bug for constructing ArrayView with fixed-length type (#28185)
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-07 23:38:21 +08:00
cai.zhang
fd68837f84
Fix bug for setting index state when indexnode connecting failed (#28220)
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-07 23:34:26 +08:00
zhenshan.cao
2057fdce43
Change default storage type to minio (#28236)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-11-07 21:15:31 +08:00