Commit Graph

32 Commits

Author SHA1 Message Date
groot
9f8095996b
Report bulkinsert progress (#21612) (#21638)
Signed-off-by: groot <yihua.mo@zilliz.com>
2023-01-11 17:37:44 +08:00
groot
aab20a4031
Streaming read numpy (#21579)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-01-10 14:53:38 +08:00
Ten Thousand Leaves
b3da554705
Increase bulk insert file size limit (#21526)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-01-06 13:09:36 +08:00
groot
33a102c6fc
Refine bulkinsert (#20986)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-12-07 14:53:18 +08:00
groot
8da44638ac
Refine json parser for import (#20545)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-16 19:05:08 +08:00
groot
53850e340c
Dont expire pending import tasks (#20431)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-10 10:25:08 +08:00
wayblink
a7bed1c927
Pass backup flag through Import request options (#20334)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-11-08 11:33:03 +08:00
groot
b847c425e1
Fix listImportTasks bug (#20301)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-07 17:11:02 +08:00
groot
cd3b3d3da6
Only allow one file for row-based import (#20234)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-02 10:23:35 +08:00
groot
c6151ad351
Parse utf32 string of numpy file (#20176)
Signed-off-by: groot <yihua.mo@zilliz.com>

Signed-off-by: groot <yihua.mo@zilliz.com>
2022-10-31 17:39:34 +08:00
groot
bee66631e3
Refine bulkload (#19671)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-10-27 16:21:34 +08:00
SimFG
a55f739608
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
Ten Thousand Leaves
8b8df0a5e9
Do not use DataCoord context when DataNode is handling import task (#19732)
So that when DataCoord is done, DataNode can still proceed.

/kind bug

issue: #19730
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-10-14 15:15:24 +08:00
groot
a561614d48
Fix bulkload failed bug (#19702)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-10-12 10:27:23 +08:00
groot
2599a3ece0
Import native binlog files (#19447)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-09-30 10:32:54 +08:00
congqixia
838a633584
Add ctx parameter for ChunkManager methods (#19546)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-29 16:18:56 +08:00
Ten Thousand Leaves
b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-09-26 18:06:54 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
groot
371a44f018
Fix a bug of bulkload (#17620)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-06-17 20:52:11 +08:00
groot
ab9b186f15
Change bulkload bufsize for row-based parser (#17473)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-06-14 16:18:09 +08:00
groot
9210299706
Fix bulkload OOM issue (#17461)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-06-09 16:52:07 +08:00
groot
d525e95550
Bulkload not allow duplicate numpy file (#17105)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-05-20 10:27:56 +08:00
groot
168935f4cf
Fix bulkload bug for string primary key (#16958)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-05-13 16:07:54 +08:00
groot
8736372fd2
Fix bulkload bugs (#16760)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-05-06 11:21:50 +08:00
groot
21b5973253
Refine list import task api (#16690)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-28 17:21:47 +08:00
groot
bc5e9ec929
Add list import tasks api (#16605)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-25 17:37:46 +08:00
groot
a6a3b69d91
Report bulk load state (#16555)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-21 21:37:42 +08:00
Ten Thousand Leaves
289e468a7a
Make bulk load fully work (#16512)
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-20 14:03:40 +08:00
godchen
4781db8a2a
Add datanode import (#16414)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-12 22:19:34 +08:00
groot
be8d9a8b6b
Support import numpy file (#16348)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-03 11:27:29 +08:00
groot
bd241cd28b
Add performance test case for import (#16296)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-01 10:07:28 +08:00
groot
ffa06c77b6
Import util functions (#16237)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-30 16:25:30 +08:00