Commit Graph

5408 Commits

Author SHA1 Message Date
congqixia
785a5a757f
Use segment version instead of ref cnt (#17609)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-17 17:38:12 +08:00
congqixia
cc3ecc4bd5
Make querycoord channel allocator respect context (#17552)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-17 16:02:12 +08:00
Jiquan Long
0e29c37499
Forbid system fields in user schema (#17613)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-17 14:10:11 +08:00
yah01
f5fa93aa0b
Return err if failed to assign segments/channels to nodes (#17616)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-17 12:42:11 +08:00
SimFG
2008f60e50
Simplify the implementations of the ToPhysicalChannel and the ConvertChannelName func (#17605)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-06-17 10:22:11 +08:00
yah01
0f87763682
Fix LoadBalance may not update the shard leader (#17608)
This happens probably with concurrent updating replicas,
some goroutines modify the nodes list of replicas,
and the others modify the shard leaders of replicas

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-17 08:58:10 +08:00
cai.zhang
53a49a9255
GetIndexState only get the state generated call CreateIndex (#17549)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-16 20:12:11 +08:00
yah01
3f42f5f345
Set the task state to TaskFailed if error occurs (#17598)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-16 19:18:11 +08:00
Jiquan Long
8e2797fe30
Print error when failed to broadcast message (#17595)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-16 18:32:10 +08:00
shaoyue
04f836f516
Fix s3 delete file (#17574)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-06-16 18:14:10 +08:00
Letian Jiang
be4077421e
Filter duplicated sealed segments which are both online and offline (#17593)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-06-16 17:42:11 +08:00
Xiaofan
1f6fbf91b2
Fix pulsar unsubsribe issue (#17562)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-16 17:28:11 +08:00
Jiquan Long
2f66531fdf
Fix wrong IP distances (#17590)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-16 16:34:11 +08:00
yah01
7d5c8c5f38
Fix bug not remove offline node (#17560)
The LoadBalance task won't remove the offline node if the node never load/watch any segment/dmchannel

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-16 16:00:10 +08:00
groot
b5e62023ea
Return error when bulkload pending list hit limit (#17570)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-06-16 13:02:10 +08:00
wayblink
eb5b0b7fc8
Move SegmentInfo out of VchannelInfo, leave Id only to decrease kv size. Get complete SegmentInfo through RPC (#17441)
Resolves: #17233 #16047

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-16 12:00:10 +08:00
Xiaofan
54aeb077e9
Refine compaction selection poliy (#17486)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-15 23:14:10 +08:00
cai.zhang
ea5041aec2
Acquiring the segment reference lock on task level (#17544)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-15 21:38:10 +08:00
Jiquan Long
875b6f88b0
Add channel information to flow graph node (#17349)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-15 21:04:10 +08:00
Ten Thousand Leaves
51a9e54b21
Fix a bug that causes infinite loop (#17550)
This will cause AddNode() to keep its lock forever so DataCoord would just hang

issue: #17366
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-15 20:50:10 +08:00
xige-16
98b32f3eba
Improve time record log in search/query (#17545)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-15 20:48:10 +08:00
xige-16
dd483beefd
Delete the log that prints a lot of deleteMsg.timestamps (#17543)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-15 20:46:14 +08:00
Bingyi Sun
d4fdfab818
Fix VDC-510 (#17554)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-06-15 19:58:10 +08:00
xing.zhao
76523b313b
Fix bug #17403 by double check collmeta prefix and alias prefix (#17548)
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: xingzhao <xing.zhao@zilliz.com>
2022-06-15 19:14:10 +08:00
SimFG
e825215c44
Remove the redundant 'for' in the TaskCondition (#17558)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-06-15 16:32:10 +08:00
Ten Thousand Leaves
65ab847815
Add segment reference lock for bulk load (#17457)
So we can support DataQueryable and DataIndexed states

issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-15 12:20:10 +08:00
Jiquan Long
e751663fdc
Fix err msg when the dimension of query vectors mismatch field (#17553)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-15 10:54:09 +08:00
congqixia
f9553970f9
Add BindContext function for querycoord task scheduler (#17531)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-14 17:12:09 +08:00
congqixia
ed3a1164dd
Handle wrapped grpc error in funcutil.IsGrpcError (#17547)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-14 17:10:10 +08:00
groot
ab9b186f15
Change bulkload bufsize for row-based parser (#17473)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-06-14 16:18:09 +08:00
bigsheeper
cdcdfa1ea5
Disable jemalloc and use malloc_trim instead (#17538)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-14 11:12:09 +08:00
xing.zhao
e9dcda161f
Add aliases return value in describeCollection task (#17516)
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: 赵星 <zhaoxing@zhaoxings-MacBook-Pro.local>
2022-06-13 19:22:10 +08:00
congqixia
a0d3f37c0d
Sync grpc DefaultKeepAliveTimeout to milvus.yaml (#17519)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-13 16:18:09 +08:00
XuanYang-cn
e6225d923b
Cancel timers while adding new timer (#17511)
See also: #17335

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-13 14:46:08 +08:00
congqixia
d1de8cabdd
Fix %w verb for error wrapping in grpcclient (#17484)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-13 12:48:08 +08:00
xige-16
16f0815184
Add time record log on search/query execution path (#17497)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-13 11:42:09 +08:00
xige-16
36ad989590
Fix segOffset grater than insert barrier when mark delete (#17444)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-10 20:02:08 +08:00
Enwei Jiao
a5b008acec
ignore getReplica's error when handle rebalanceTask (#17469)
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
2022-06-10 14:50:08 +08:00
jaime
b64a467311
Revert "Use unified catalog interface to reconstruct metastore (#17042)" (#17480)
This reverts commit 218326bafe.

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-06-10 13:10:08 +08:00
bigsheeper
2d9a52206d
Use jemalloc in QueryNode, DataNode and IndexNode (#17470)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-10 11:24:08 +08:00
XuanYang-cn
2b5340581c
Skip remove if reassigns to the original node (#17450)
Fix ut race

See also: #15966, #17432

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-10 10:54:07 +08:00
Xiaofan
66f26943f8
Add Err code when task rollback (#17472)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-09 21:08:07 +08:00
Xiaofan
05a918e0ab
Refine logs (#17438)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-09 21:04:07 +08:00
congqixia
97a871cc82
Make querycoord segment allocator respect context (#17452)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-09 20:14:06 +08:00
Letian Jiang
dfaed5acdd
Add QueryServiceAvailable field in ShowCollections (#17456)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-06-09 18:20:07 +08:00
yah01
a38790f586
Predict the loading memory usage more accurate (#17451)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-09 18:16:07 +08:00
Jiquan Long
2ca81620ea
Reduce lock operations when get dml stream (#17468)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-09 17:34:09 +08:00
groot
9210299706
Fix bulkload OOM issue (#17461)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-06-09 16:52:07 +08:00
xige-16
6d078ebe7a
Print more information when loading fields with different lines (#17448)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-08 20:16:06 +08:00
wayblink
25883840b2
Remove delete log in datanode because it may influence datanode performance (#17415)
Resolves #16984

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-08 19:10:07 +08:00