Commit Graph

1096 Commits

Author SHA1 Message Date
congqixia
566f5b5766
Make querynode handle segmentChangeInfo with multiple vchannel (#18009)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-01 19:20:20 +08:00
bigsheeper
641fb96958
Return malloc_trim result and log if malloc_trim successfully (#17959)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-01 10:44:18 +08:00
congqixia
f4177f4434
Remove growing according to offline as well (#17968)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-30 20:14:19 +08:00
zhenshan.cao
0baeb609dc
Fix bugs in monitoring metrics (#17894)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-30 17:26:19 +08:00
congqixia
c179986e6c
Fix shard cluster segments data race (#17903)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-29 16:20:19 +08:00
bigsheeper
6b79d4f7bc
Log segment type as possible in QueryNode (#17846)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-28 10:16:17 +08:00
yah01
797218a8ad
Fix handoff isn't processed correctly (#17738)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-27 22:08:17 +08:00
congqixia
68d385137a
Fix: Miss channel info when load growing segment (#17789) (#17797)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: zhagnlu <1542303831@qq.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-06-27 12:02:16 +08:00
congqixia
5471e35ccc
Wait last version search to release growing safely (#17713)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-25 18:48:15 +08:00
Enwei Jiao
16c3aedc15
refine complie configuration (#17502)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-06-24 21:12:15 +08:00
xige-16
3a0f9d41df
Print log after segment search done (#17769)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-24 14:10:14 +08:00
codeman
294f7e4408
fix: log datapb.SegmentInfo is too large #17685 (#17727)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-06-23 14:36:13 +08:00
Jiquan Long
216e2f80aa
Add msg id to log in query path (#17677)
/kind improvement

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-23 10:46:13 +08:00
codeman
6cb00aa5bb
fix: log datapb.SegmentInfo is too large #17685 (#17693)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-06-22 21:46:13 +08:00
bigsheeper
8cf54137cf
Add nil check in reduce and add collection rwmutex when release segments (#17698)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-22 16:12:14 +08:00
Jiquan Long
0282f69eb4
Add msg id to log in search path. (#17654)
issue: #17513
Add msg id to log in search path,
which should help us to trace a detailed request more easier.

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-21 13:30:12 +08:00
congqixia
aa4b42c74b
Remove wait before release collection/partition (#17623)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-17 19:00:11 +08:00
bigsheeper
92d06b2e30
Purge memory by the memory state and try to purge after each search (#17565)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-17 17:46:10 +08:00
congqixia
785a5a757f
Use segment version instead of ref cnt (#17609)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-17 17:38:12 +08:00
Letian Jiang
be4077421e
Filter duplicated sealed segments which are both online and offline (#17593)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-06-16 17:42:11 +08:00
wayblink
eb5b0b7fc8
Move SegmentInfo out of VchannelInfo, leave Id only to decrease kv size. Get complete SegmentInfo through RPC (#17441)
Resolves: #17233 #16047

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-16 12:00:10 +08:00
Jiquan Long
875b6f88b0
Add channel information to flow graph node (#17349)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-15 21:04:10 +08:00
xige-16
98b32f3eba
Improve time record log in search/query (#17545)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-15 20:48:10 +08:00
xige-16
dd483beefd
Delete the log that prints a lot of deleteMsg.timestamps (#17543)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-15 20:46:14 +08:00
xige-16
16f0815184
Add time record log on search/query execution path (#17497)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-13 11:42:09 +08:00
yah01
a38790f586
Predict the loading memory usage more accurate (#17451)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-09 18:16:07 +08:00
yah01
70f8bea4b4
Avoid growing slice as deserializing binlogs (#17421)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-08 11:46:06 +08:00
zhenshan.cao
98e95275fe
Fix several code issues caused by carelessness (#17385)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-06 18:58:05 +08:00
congqixia
ac6394d0fd
Fix container/list removal in iteration (#17382)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-06 16:34:05 +08:00
Xiaofan
af994b5e1c
Fix etcd watch error (#17364)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-06 16:26:06 +08:00
bigsheeper
45febac298
Extract lease of cProto in QueryNode (#17373)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-06 15:56:05 +08:00
Ten Thousand Leaves
f5e63177d2
Fix an issue with bulk load with multiple DataNodes (#17324)
issue: #17300
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-02 18:54:04 +08:00
yah01
4e83f37899
Improve load performance (#17273)
Load segment's fields concurrently

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-02 17:56:03 +08:00
xige-16
8c69790383
Fix lost delete msg caused by loadSegment after watchDeltaChannel (#17308)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-02 16:56:04 +08:00
zhenshan.cao
5f44e45480
Add metrics on QueryNode and Proxy (#17328)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-02 16:06:03 +08:00
bigsheeper
7a59a80506
Remove segmentPtr lock and use collection lock instead (#17303)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-02 14:00:04 +08:00
congqixia
c76b4ade01
Fix data race in LoadSegmentsTask (#17342)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-02 12:52:03 +08:00
zhenshan.cao
2763efc9b0
Add cache of grpc client of ShardLeader in proxy (#17301)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-02 12:16:03 +08:00
xige-16
63a31ccb38
Remove error return value when removeSegment (#17310)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-01 13:18:02 +08:00
bigsheeper
f55fe6be3c
Merge historical replica and streaming replica in QueryNode (#17217)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-31 13:42:03 +08:00
congqixia
c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-30 19:50:04 +08:00
congqixia
cecbd40a22
Fix Data race of loadType in querynode/collection.go (#17275)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-30 19:16:02 +08:00
bigsheeper
11834a0948
Improve query log in QueryNode (#17268)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-30 17:18:03 +08:00
Jiquan Long
ac2cadc83b
Refine segment type logs to string (#17248)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-30 15:30:01 +08:00
zhenshan.cao
ad9276a440
Add metrics for proxy and querynode (#17205)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-27 14:12:01 +08:00
xige-16
e196caf604
Force reclamation of memory after loading (#17244)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-27 12:50:00 +08:00
xige-16
56778787be
Reverse data from scalar index (#17145)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-26 14:58:01 +08:00
zhenshan.cao
ec1103ca27
Simplify the merge logic of searchTask (#17194)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 21:27:59 +08:00
bigsheeper
a4ea2fb18a
Handle errors in DataNode and QueryNode flow graph (#17096)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-24 21:11:59 +08:00
zhenshan.cao
d05c24a0ff
Delete unused code (#17182)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 14:49:59 +08:00