Commit Graph

1035 Commits

Author SHA1 Message Date
Xiaofan
7d51b65260
Estimate varchar length more reasonablly (#17688)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-22 13:02:13 +08:00
SimFG
eb1f0bc805
Add the consistency check tool (#17136)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-06-21 10:28:12 +08:00
groot
371a44f018
Fix a bug of bulkload (#17620)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-06-17 20:52:11 +08:00
bigsheeper
92d06b2e30
Purge memory by the memory state and try to purge after each search (#17565)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-17 17:46:10 +08:00
SimFG
2008f60e50
Simplify the implementations of the ToPhysicalChannel and the ConvertChannelName func (#17605)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-06-17 10:22:11 +08:00
cai.zhang
53a49a9255
GetIndexState only get the state generated call CreateIndex (#17549)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-16 20:12:11 +08:00
Xiaofan
1f6fbf91b2
Fix pulsar unsubsribe issue (#17562)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-16 17:28:11 +08:00
Xiaofan
54aeb077e9
Refine compaction selection poliy (#17486)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-15 23:14:10 +08:00
Ten Thousand Leaves
65ab847815
Add segment reference lock for bulk load (#17457)
So we can support DataQueryable and DataIndexed states

issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-15 12:20:10 +08:00
congqixia
ed3a1164dd
Handle wrapped grpc error in funcutil.IsGrpcError (#17547)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-14 17:10:10 +08:00
groot
ab9b186f15
Change bulkload bufsize for row-based parser (#17473)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-06-14 16:18:09 +08:00
congqixia
a0d3f37c0d
Sync grpc DefaultKeepAliveTimeout to milvus.yaml (#17519)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-13 16:18:09 +08:00
congqixia
d1de8cabdd
Fix %w verb for error wrapping in grpcclient (#17484)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-13 12:48:08 +08:00
yah01
a38790f586
Predict the loading memory usage more accurate (#17451)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-09 18:16:07 +08:00
groot
9210299706
Fix bulkload OOM issue (#17461)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-06-09 16:52:07 +08:00
xige-16
b5c11a216d
Alter varChar type params's name to max_length (#17409)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-07 15:58:06 +08:00
Xiaofan
a9ec1f8864
Fix compatibility of upgrade (#17406)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-07 14:52:06 +08:00
congqixia
de8f2f76c5
Fix shardLeader cache with ErrConnect check (#17402)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-07 12:20:06 +08:00
Xiaofan
af994b5e1c
Fix etcd watch error (#17364)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-06 16:26:06 +08:00
XuanYang-cn
c70af73005
Fix datacoord set wrong state for node registering (#17376)
Fix datacoord datarace

See also: #17335

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-06 15:10:05 +08:00
bigsheeper
48cf63a2d7
Wait for flow graph to exit (#17341)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-02 19:48:04 +08:00
shaoyue
76eaa3fc50
Support access s3 through AWS IAM role (#17292)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-06-02 19:42:03 +08:00
SimFG
0986c29d7f
proxy tls support one-way authentication (#17348)
Signed-off-by: xiyichan <2863768433@qq.com>

Co-authored-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>

Co-authored-by: xiyichan <2863768433@qq.com>
2022-06-02 18:56:04 +08:00
Ten Thousand Leaves
f5e63177d2
Fix an issue with bulk load with multiple DataNodes (#17324)
issue: #17300
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-02 18:54:04 +08:00
yah01
4e83f37899
Improve load performance (#17273)
Load segment's fields concurrently

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-02 17:56:03 +08:00
zhenshan.cao
2763efc9b0
Add cache of grpc client of ShardLeader in proxy (#17301)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-02 12:16:03 +08:00
jaime
5698bf4236
Add sasl configurations for kafka (#17323)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-06-02 12:12:03 +08:00
cai.zhang
bcf3b7426a
Add distributed lock for segment refgerence (#16782)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
Jiquan Long
6bd786bf50
Reduce default seal propotion (#17287)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-31 12:02:04 +08:00
congqixia
c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-30 19:50:04 +08:00
zhenshan.cao
b0d54ed0af
Rename Client to GrpcClient in mock package (#17230)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-30 17:16:08 +08:00
SimFG
731870211a
Support role-based access control (only pb) (#16987)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-05-28 00:04:01 +08:00
Ten Thousand Leaves
5e1e7a6896
Fix an issue that causes flush hanging (#17169)
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-25 14:34:00 +08:00
congqixia
a0cef3a0a2
Fix sessionutil log always datanode (#17184)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-24 16:05:59 +08:00
zhenshan.cao
2512e668f7
Update timestmap according to gracefultime for Bounded Consistency (#17171)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 12:05:59 +08:00
zhenshan.cao
cce533c246
Fix unitest fail, change default index slice to 16 (#17162)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-23 17:44:25 +08:00
zhenshan.cao
08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
ownbylichaobao
2f8a7e7793
Support tls/ssl connection to etcd (#7703) (#17012)
Signed-off-by: lichaobao <1527563274@qq.com>
2022-05-20 12:29:19 +08:00
groot
d525e95550
Bulkload not allow duplicate numpy file (#17105)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-05-20 10:27:56 +08:00
bigsheeper
9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID (#16882)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-19 10:13:56 +08:00
Xiaofan
5355153805
Minio Error is not Handled gracefully (#17003)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-16 19:25:55 +08:00
groot
168935f4cf
Fix bulkload bug for string primary key (#16958)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-05-13 16:07:54 +08:00
exia
4ee78eb5ad
proxy support tls (#16706)
Signed-off-by: xiyichan <2863768433@qq.com>
2022-05-10 20:05:53 +08:00
Bingyi Sun
76bfd41952
Refine DataNode logs. (#16849)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-05-09 16:25:52 +08:00
Enwei Jiao
67ce206cf0
make SegmentMaxLifetime configurable (#16783)
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
2022-05-06 23:41:51 +08:00
Jiquan Long
98ceb162aa
Support antlr as plan parser (#16696)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: xaxys <tpnnghd@163.com>

Co-authored-by: xaxys <tpnnghd@163.com>
2022-05-06 17:43:51 +08:00
groot
8736372fd2
Fix bulkload bugs (#16760)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-05-06 11:21:50 +08:00
Ten Thousand Leaves
1acd256481
Add DataQueryable and DataIndexed states for bulk load tasks (#16725)
issue: #16607

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-05 21:17:50 +08:00
Xiaofan
333b439a6c
Support config grpc log level (#16618)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-05 10:35:50 +08:00
shaoyue
fa97f86cda
Clean up not used id_cache related code (#16655)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-05-05 09:59:51 +08:00