Commit Graph

40 Commits

Author SHA1 Message Date
Ten Thousand Leaves
e66ac6a77c
Resolve an issue with bulk load where segments in flow graph replica are not updated (#16609)
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-25 11:07:47 +08:00
congqixia
69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470)
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-20 16:15:41 +08:00
Ten Thousand Leaves
289e468a7a
Make bulk load fully work (#16512)
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-20 14:03:40 +08:00
Ten Thousand Leaves
442c73a544
Support passing channel names to DataNode (#16424)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-12 13:25:34 +08:00
Ten Thousand Leaves
1e6f6bd072
Implement set segment state call from RootCoord to DataCoord. (#16374)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-06 15:33:32 +08:00
Ten Thousand Leaves
45be3deb3f
Implement RC, DC, DN calling path for import. (#16321)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-01 11:33:28 +08:00
Ten Thousand Leaves
0d54697d42
Implement ReportImport related logic. (#16312)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-31 13:51:28 +08:00
Ten Thousand Leaves
a23ca745bb
Add Etcd support for Import Tasks. (#16164)
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-25 11:03:25 +08:00
groot
1c4b949a1d
modify import rpc interface (#16129)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-22 15:11:24 +08:00
groot
89356ebcb9
Modify import rpc interfaces (#16006)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-11 17:13:59 +08:00
groot
5fdef607d5
Add rpc interfaces for import (#15930)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-09 18:33:59 +08:00
congqixia
54862a31b0
Extend ChannelWatchState in protobuf (#15847)
Related #15846
/kind enhancement

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-03 18:43:56 +08:00
congqixia
5751759c85
Implement mix compaction logic (#15542)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-18 14:51:49 +08:00
congqixia
1dcd06cc78
Add log file info for all kind of segment binlogs (#13606)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-19 20:00:42 +08:00
Bingyi Sun
0c39e0370c
Move segment statistics update to datanode tt loop (#13233)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:53:16 +08:00
congqixia
06ced4d1d1
Add DropVirtualChannel for DataCoord (#12361)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-29 22:35:41 +08:00
bigsheeper
e245ba87a9
Add excluded segments for dropped segemnts (#12247)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-25 14:25:15 +08:00
congqixia
8e0c8febe9
Add dropped segment ids in vchaninfo (#12230)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-23 18:59:15 +08:00
Bingyi Sun
01be91116e
Add GetFlushState interface (#12170)
issue: #12075
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-23 10:55:14 +08:00
congqixia
7e43b568ee
Add dropped ts and apply when gc scan dropped segments (#11838)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-16 14:23:21 +08:00
godchen
5357e301f4
Fix retrieve search error (#11501)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-11 00:54:45 +08:00
congqixia
e5f599b616
Add dropped flag in SaveBinlogPathRequest (#11519)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-09 22:27:37 +08:00
Bingyi Sun
c3f0c5a3ff
Add compaction interfaces in proxy (#11431)
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-09 14:47:02 +08:00
sunby
c26cd79532
Add compaction (#9906)
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-11-05 22:25:00 +08:00
godchen
d6bf0c928f
Get stats delta path from datacoord (#10362)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-22 14:31:13 +08:00
congqixia
7ce7cb7a5e
Add a timed flush trigger mechanism (#10197)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-20 15:02:36 +08:00
sunby
54459ac3b3
Add compaction proto (#9874)
issue: #9873
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-20 11:58:35 +08:00
congqixia
a7f7bff651
Modify SaveBinlogPaths proto to support delete logs (#10080)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-19 00:26:35 +08:00
XuanYang-cn
40d3562ec0
Fix comment on SegmentInfo message (#10066)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-18 21:27:09 +08:00
XuanYang-cn
98ed253e2c
Update SegmentInfo proto to support deltalogs (#9925)
Resolves: #9922
See also: #9530

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-15 18:24:39 +08:00
godchen
937dbf4279
Add flushed segments (#9476)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-08 19:09:12 +08:00
congqixia
ffbfaf7f76
Apply collection start position when open segment (#8697)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-27 20:34:03 +08:00
bigsheeper
b7103357fd
Load failed if collection beyonds the memory limit (#7434)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-07 11:35:18 +08:00
dragondriver
89748ac639
Expose metrics of data cluster (#7177)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-01 10:13:15 +08:00
sunby
4d56d53c0b
Add a tool to move binlog path to SegmentInfo (#7243)
issue: #7214
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-24 15:51:51 +08:00
sunby
4f7c3e7d3a
Save binlog path in SegmentInfo (#6590)
issue: #6589
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-19 13:00:12 +08:00
congqixia
e1b3273d1c
Add Datanode watch etcd channel (#6965)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-13 10:50:09 +08:00
sunby
07b0989628
Add GetFlushedSegments in data coordinator (#6253)
Send segment flush message by grpc and add a new interface
`GetFlushedSegments`

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-02 11:16:20 +08:00
sunby
46f86a0354
Return sealed segments when calling flush (#6011)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-23 16:56:11 +08:00
sunby
c863a193de
Rename filename from dataservice to datacoordinator (#5946)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-22 10:42:07 +08:00