Commit Graph

33 Commits

Author SHA1 Message Date
yihao.dai
0fe5e90e8b
enhance: Remove import v1 (#31403)
Remove all code and logic related to import v1.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-22 15:29:09 +08:00
aoiasd
0c153a5820
enhance: Rename update segment operator (#31121)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-20 17:53:14 +08:00
yihao.dai
776709e5ff
fix: Fix binlog import (#31310)
Fix binlog import functionality by removing the existing check and
refining the size retrieval process.

issue: https://github.com/milvus-io/milvus/issues/31221,
https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-17 20:59:04 +08:00
yihao.dai
811316d2ba
fix: Fix binlog import and refine error reporting (#31241)
1. Fix binlog import with partition key.
2. Refine binlog import error reportins.
3. Avoid division by zero when retrieving import progress.

issue: https://github.com/milvus-io/milvus/issues/31221,
https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-15 10:55:05 +08:00
jaime
4b0c3dd377
enhance: index meta use independent rather than global meta lock (#30869)
issue: https://github.com/milvus-io/milvus/issues/30837

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-03-04 16:56:59 +08:00
yihao.dai
a434d33e75
feat: Add import scheduler and manager (#29367)
This PR introduces novel managerial roles for importv2:
1. ImportMeta: To manage all the import tasks;
2. ImportScheduler: To process tasks and modify their states;
3. ImportChecker: To ascertain the completion of all tasks and instigate
relevant operations.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-01 18:31:02 +08:00
aoiasd
2180e2cfae
fix: wrong segment binlog path cause load segment failed #30726 (#30959)
DataCoord GetSegmentInfo should return binlog info with logpath instead
logid when segment merge child segment's
binlog.
relate: https://github.com/milvus-io/milvus/issues/30366
https://github.com/milvus-io/milvus/issues/30165
https://github.com/milvus-io/milvus/issues/30550

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-01 13:09:01 +08:00
XuanYang-cn
e184c891ff
fix: [skip-e2e] Fix unstable SaveBinlogPath ut (#30508)
Fixes: #30507

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-02-05 16:00:59 +08:00
XuanYang-cn
6959630652
fix: donot set l0 segment as growing when savebinlogs (#29194)
This PR fixes negative growing L0 segments in Metrics

See also: #29204, #30441

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-02-04 10:21:06 +08:00
yihao.dai
7ce876a072
fix: Decoupling importing segment from flush process (#30402)
This pr decoups importing segment from flush process by:
1. Exclude the importing segment from the flush policy, this approch
avoids notifying the datanode to flush the importing segment, which may
not exist.
2. When RootCoord call Flush, DataCoord directly set the importing
segment state to `Flushed`.

issue: https://github.com/milvus-io/milvus/issues/30359

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-02-03 13:01:12 +08:00
smellthemoon
e52ce370b6
enhance:don't store logPath in meta to reduce memory (#28873)
don't store logPath in meta to reduce memory, when service get
segmentinfo, generate logpath from logid.
#28885

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-01-18 22:06:31 +08:00
XuanYang-cn
75e6b65c60
enhance: Use ChannelManger interface in Server (#29629)
See also: #29447

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-01-08 17:46:47 +08:00
congqixia
8a63e53421
enhance: Add http method to control datacoord garbage collection (#29052)
See also #29051

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-14 19:26:39 +08:00
XuanYang-cn
a153950b10
Change channel to Interface (#27839)
This PR changes `*channel` into RWChannel interface

See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-13 11:16:18 +08:00
wei liu
7e0bda8b4c
remove binlog/delta log from getRecoveryInfoV2 (#27895)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-30 14:28:15 +08:00
Filip Haltmayer
6b1a106a31
Moving etcd client into session (#27069)
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
2023-10-27 07:36:12 +08:00
Xiaofan
2ea7579dbb
Reduce rpc size for GetRecoveryInfoV2 (#27483)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-10-23 21:44:09 +08:00
congqixia
49516d44b4
Add ctx parameter and log tracer for watch and selectNodes (#27809)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-20 04:22:11 +08:00
MrPresent-Han
cb71a3e235
rm dependency to rc when getting recovery info(#25363) (#27405)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-10-09 18:51:32 +08:00
yah01
6539a5ae2c
Refine DataCoord status (#27262)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 17:15:27 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface (#27186)
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
MrPresent-Han
4f6122dd4a
enable ctx traceId for assignsegment on dc(#26972) (#27108)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-09-18 09:53:22 +08:00
yah01
168e82ee10
Fix panic while handling with the nil status (#27040)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-15 10:09:21 +08:00
aoiasd
b533c68632
Forbid update checkpoint without msgID (#25694)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-07-25 10:43:04 +08:00
congqixia
597a4d9227
Treat small segment without index as sealed (#25237)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-02 19:50:23 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yah01
ebd0279d3f
Check error by Error() and NoError() for better report message (#24736)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
wei liu
cbfe7a45ef
fix pull target (#23491)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-18 18:30:32 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
Jiquan Long
9fd9bed2b9
Remove collection meta after GC finished (#21595)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-12 09:55:42 +08:00
cai.zhang
e5f408dceb
Merge IndexCoord and DataCoord (#21267)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-04 19:37:36 +08:00
jaime
0a5270e8c0
Fix alter collection hang (#20694)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-11-21 10:09:10 +08:00