Commit Graph

68 Commits

Author SHA1 Message Date
jaime
10c03de628
Refine method name from BroadCastXx to BroadcastXx (#19700)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-11 21:07:23 +08:00
jaime
31db01b6ae
Support TTL for collection level (#19383)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-10 20:31:22 +08:00
wayblink
85e04d845c
Move GetComponentStates to milvus.proto, make it an external interface (#19594)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-10-10 15:55:22 +08:00
congqixia
d64f318a80
Add collection schema in watch channel info (#19663)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-10 01:37:21 +08:00
wayblink
e7c2500c71
Implement GetPersistentSegmentInfo through datacoord (#19205)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-10-08 11:51:02 +08:00
XuanYang-cn
a0ce533727
Add a SyncSegments to sync meta between DN and DC (#19298)
See also: #19072

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-27 16:02:53 +08:00
Ten Thousand Leaves
b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-09-26 18:06:54 +08:00
jaime
91df8f2d6a
Use log id instead of log path of the binlog in metastore (#19123)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-09-25 15:56:51 +08:00
XuanYang-cn
432b5ddf2e
Add SyncSegments into proto (#19312)
See also: #19072

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-23 10:22:52 +08:00
cai.zhang
fc3309b8b8
All indexes segments both write handoff (#19230)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-21 16:34:51 +08:00
wayblink
3927ae9952
Return all flushed segments and time-of-seal in Flush interface (#18882)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-09-09 09:58:37 +08:00
jaime
362c663b22
Remove deprecated inner compaction type (#19024)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-09-07 19:16:34 +08:00
Enwei Jiao
d9c7519107
refactor compaction concurency logic (#18660)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-23 15:50:52 +08:00
Jiquan Long
641afae22f
Watch channels with start positions (#18744)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-08-22 14:42:52 +08:00
codeman
70254c3ec5
Unified catalog interface for segment (#18289) (#18290)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-08-20 10:24:51 +08:00
aoiasd
8a70f2badb
Add ShowConfigurations for all remaining session (#18594)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-08-12 13:20:39 +08:00
Jeng.Gwan
638f6c36e9
Support to get real row count of segment (#18115)
Signed-off-by: xaxys <zheng.guan@zilliz.com>
2022-07-18 09:58:28 +08:00
xing.zhao
3904a78211
Remove genReqID function in import manager bacause the request ID is useless (#17853)
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: xingzhao <xing.zhao@zilliz.com>
2022-06-28 18:20:18 +08:00
Ten Thousand Leaves
f0b036a35a
Move bulk load segment lock happen early (#17612)
issue: #17600
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-27 13:56:17 +08:00
wayblink
074ec3060a
Support return dropped segments info in GetSegmentInfo rpc (#17617)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-17 18:24:12 +08:00
wayblink
eb5b0b7fc8
Move SegmentInfo out of VchannelInfo, leave Id only to decrease kv size. Get complete SegmentInfo through RPC (#17441)
Resolves: #17233 #16047

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-16 12:00:10 +08:00
cai.zhang
ea5041aec2
Acquiring the segment reference lock on task level (#17544)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-15 21:38:10 +08:00
Ten Thousand Leaves
65ab847815
Add segment reference lock for bulk load (#17457)
So we can support DataQueryable and DataIndexed states

issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-15 12:20:10 +08:00
groot
f31be941bb
Fix bulkload row count bug for multi datanodes (#17384)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-06-07 16:56:07 +08:00
Ten Thousand Leaves
f5e63177d2
Fix an issue with bulk load with multiple DataNodes (#17324)
issue: #17300
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-02 18:54:04 +08:00
cai.zhang
bcf3b7426a
Add distributed lock for segment refgerence (#16782)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
Ten Thousand Leaves
5e1e7a6896
Fix an issue that causes flush hanging (#17169)
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-25 14:34:00 +08:00
Ten Thousand Leaves
63385b7c12
Make data queryable and data indexed states as independent flags (#16837)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-12 19:23:53 +08:00
Ten Thousand Leaves
e66ac6a77c
Resolve an issue with bulk load where segments in flow graph replica are not updated (#16609)
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-25 11:07:47 +08:00
congqixia
69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470)
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-20 16:15:41 +08:00
Ten Thousand Leaves
289e468a7a
Make bulk load fully work (#16512)
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-20 14:03:40 +08:00
Ten Thousand Leaves
442c73a544
Support passing channel names to DataNode (#16424)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-12 13:25:34 +08:00
Ten Thousand Leaves
1e6f6bd072
Implement set segment state call from RootCoord to DataCoord. (#16374)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-06 15:33:32 +08:00
Ten Thousand Leaves
45be3deb3f
Implement RC, DC, DN calling path for import. (#16321)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-01 11:33:28 +08:00
Ten Thousand Leaves
0d54697d42
Implement ReportImport related logic. (#16312)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-31 13:51:28 +08:00
Ten Thousand Leaves
a23ca745bb
Add Etcd support for Import Tasks. (#16164)
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-25 11:03:25 +08:00
groot
1c4b949a1d
modify import rpc interface (#16129)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-22 15:11:24 +08:00
groot
89356ebcb9
Modify import rpc interfaces (#16006)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-11 17:13:59 +08:00
groot
5fdef607d5
Add rpc interfaces for import (#15930)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-09 18:33:59 +08:00
congqixia
54862a31b0
Extend ChannelWatchState in protobuf (#15847)
Related #15846
/kind enhancement

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-03 18:43:56 +08:00
congqixia
5751759c85
Implement mix compaction logic (#15542)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-18 14:51:49 +08:00
congqixia
1dcd06cc78
Add log file info for all kind of segment binlogs (#13606)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-19 20:00:42 +08:00
Bingyi Sun
0c39e0370c
Move segment statistics update to datanode tt loop (#13233)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:53:16 +08:00
congqixia
06ced4d1d1
Add DropVirtualChannel for DataCoord (#12361)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-29 22:35:41 +08:00
bigsheeper
e245ba87a9
Add excluded segments for dropped segemnts (#12247)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-25 14:25:15 +08:00
congqixia
8e0c8febe9
Add dropped segment ids in vchaninfo (#12230)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-23 18:59:15 +08:00
Bingyi Sun
01be91116e
Add GetFlushState interface (#12170)
issue: #12075
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-23 10:55:14 +08:00
congqixia
7e43b568ee
Add dropped ts and apply when gc scan dropped segments (#11838)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-16 14:23:21 +08:00
godchen
5357e301f4
Fix retrieve search error (#11501)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-11 00:54:45 +08:00
congqixia
e5f599b616
Add dropped flag in SaveBinlogPathRequest (#11519)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-09 22:27:37 +08:00