Commit Graph

98 Commits

Author SHA1 Message Date
bigsheeper
772771bd85
Stop queryCollection before releasing replica (#12982)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-12-09 14:31:28 +08:00
bigsheeper
496f3d0009
Move AddQueryChannel to taskScheduler in query node (#12294)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-30 10:29:43 +08:00
bigsheeper
65608836e6
Remove vChannel for collection in historical (#12359)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-29 20:11:49 +08:00
bigsheeper
58964621a5
Remove tSafeWatcher and vChannel when release (#12264)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-26 01:33:16 +08:00
godchen
88012acb86
Clear useless segments info. (#12262)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-25 17:37:15 +08:00
bigsheeper
e245ba87a9
Add excluded segments for dropped segemnts (#12247)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-25 14:25:15 +08:00
bigsheeper
62bbc9cd1d
Add excluded segments for flushed segments in query node (#12188)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-24 11:27:15 +08:00
godchen
116a503b4a
From dml cp load delete msg (#12144)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-21 07:33:14 +08:00
Bingyi Sun
e3e4dc7153
Improve GetVChanPosistion and fix unit tests (#11888)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-16 21:41:50 +08:00
godchen
4cd7680f87
Reschedule load segments (#11391)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-13 08:49:08 +08:00
bigsheeper
93149c5ad9
Load growing segment in query node (#11664)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-12 18:27:10 +08:00
godchen
5357e301f4
Fix retrieve search error (#11501)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-11 00:54:45 +08:00
godchen
639800241d
Make flow graph for delta channel (#11379)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-09 09:27:04 +08:00
yukun
4e5c8811da
Add flow_graph for delta channel in querynode (#11293)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-11-06 11:02:58 +08:00
bigsheeper
5c58924420
Add cgroups to get container memory and check index memory in segment loader (#10363)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-22 20:49:10 +08:00
xige-16
8a046a6495
BroadCast sealedSegmentChangeInfo to queryChannel (#10084)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-22 19:07:15 +08:00
groot
70eb275ef4
[skip ci] Fix typo (#10412)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-10-22 14:11:52 +08:00
bigsheeper
4c420dac21
Update excluded segments info when receive changeInfo (#10011)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-16 17:00:33 +08:00
bigsheeper
e7d92e4e79
Add streaming segments memory to used memory when checkLoadMemory (#8940)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-30 21:54:08 +08:00
bigsheeper
977d55b66c
[skip ci] Add comment for task (#8497)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-24 20:53:49 +08:00
bigsheeper
52126f2d5a
Dynamic add tSafe watcher, use sync.Cond instead of selectCase, add ref count (#8050)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-24 13:57:54 +08:00
bigsheeper
c9b6a92366
Add release failed case unittest for query node (#7971)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-17 11:35:49 +08:00
bigsheeper
14391fe9a2
Release flowgraphs properly (#7266)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-08-27 17:03:56 +08:00
bigsheeper
4123deef9e
Return error message in grpc.LoadSegment in query node (#6908)
* return load error

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* init streaming meta in loadSegment

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix excluded init

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* improve impl

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-08-03 22:01:27 +08:00
bigsheeper
e8ea8b51d5
improve query node log (#6897)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-31 10:47:22 +08:00
bigsheeper
e50d39dfe1
Query result get global sealed segments from ETCD (#6377)
* get global sealed segments from ETCD

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix race error

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-14 11:15:54 +08:00
xige-16
6b65529086
Fix search error after releasePartition (#6135)
* fix search error after releasePartition

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-26 16:08:11 +08:00
bigsheeper
e87372761a
Remove tSafe in release partition task and return max timestamp (#6078)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-24 19:06:34 +08:00
yukun
b72e4c6372
Merge search_collection and retrieve_colletion into query_collection (#6037)
* Merge search_collection and retrieve_colletion into query_collection

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix static-check

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-23 20:26:10 +08:00
bigsheeper
dca24f313a
fix search timeout error (#6027)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-23 18:04:22 +08:00
bigsheeper
47d89abaa0
get channels by static converting (#6015)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-23 17:26:09 +08:00
Cai Yudong
92e429d812
Rename IndexService to IndexCoord (#5932)
* rename package indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix queryservice static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move distributed/indexservice to distributed/indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move internal/indexservice to internal/indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename MasterComponent to RootCoordComponent

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for queryservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for dataservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for datanode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for proxynode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for querynode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename IndexService to IndexCoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename IndexService to IndexCoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix rebase issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-21 17:28:03 +08:00
bigsheeper
bbd8a7e13a
Add released partitions and fix search error in empty partition (#5893)
* rename flowgraphType to loadType

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* add load type and released partitions

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* filter released partition

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-19 18:38:07 +08:00
bigsheeper
46151d203f
Fix tSafe cannot update correctly (#5820)
* Fix tSafe cannot update correctly

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* skip test_load_partitions_release_collection

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* skip test_release_collection_during_searching

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-17 16:56:04 +08:00
bigsheeper
d6f7fbf19f
release collection synchronously (#5798)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-16 15:31:57 +08:00
bigsheeper
bf39644457
Fix channels in query node and fix search (#5795)
* fix add channel error

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix tSafe vChannel key

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-16 12:05:56 +08:00
xige-16
97049ce293
use dataservice's getRecoveryInfo to get load and watch info (#5790)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-16 11:09:56 +08:00
bigsheeper
58bf92a407
Add vChannel,pChannel in query node and get vChannel from master in query service (#5784)
* add vChannel,pChannel in query node and get vChannel from master in query service

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-15 20:06:10 +08:00
zhenshan.cao
33a43f48c8 [skip ci] Fix bug: Search timeout and refactor client init logic (#5735)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:24:48 +08:00
bigsheeper
cdbc6d2c94
Refactor query node and query service (#5751)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 12:41:40 +08:00
bigsheeper
8aae0f7cc9
Refactor flow graph and load/watchDML in query node (#5682)
* Refactor flow graph and load/watchDML in query node

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-09 11:37:55 +08:00
Cai Yudong
b414800d49
Update Seek interface (#5492)
* update Seek

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update Seek for mqTtMsgStream

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* use Retry in Seek

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-29 23:21:34 +08:00
bigsheeper
03f90ff28c
refactor collection replica and add tsafe replica in querynode (#5475)
* refactor collection replica and add tsafe replica in querynode

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* static check

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* static check2

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* code format

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-05-28 15:40:32 +08:00
xige-16
fce792b8bf
Add historical and streaming module in querynode (#5469)
* add historical and streaming

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix GetSegmentInfo

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* pass regression test

Signed-off-by: xige-16 <xi.ge@zilliz.com>

Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2021-05-28 10:26:30 +08:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988)
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
bigsheeper
71b8f309b2 Add licence to query node and query service
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-19 13:47:10 +08:00
bigsheeper
9e7559b865 Make query node asynchronously load and release collection
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-16 14:40:33 +08:00
bigsheeper
39458697c7 Use asynchronous functions of load and release
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-12 09:18:43 +08:00