2020-11-19 10:59:10 +08:00
2020-12-27 09:05:24 +08:00
## 8. Query Service
2020-11-19 10:59:10 +08:00
2021-01-08 17:20:59 +08:00
2020-11-19 10:59:10 +08:00
#### 8.1 Overview
2021-01-04 14:16:43 +08:00
< img src = "./figs/query_service.jpeg" width = 700 >
2020-11-19 10:59:10 +08:00
2020-12-27 09:05:24 +08:00
#### 8.2 API
```go
type Client interface {
2020-12-29 18:02:44 +08:00
DescribeService() (ServiceDescription, error)
DescribeParition(req DescribeParitionRequest) (PartitionDescriptions, error)
LoadPartitions(req LoadPartitonRequest) error
ReleasePartitions(req ReleasePartitionRequest) error
CreateQueryChannel() (QueryChannels, error)
GetTimeTickChannel() (string, error)
GetStatsChannel() (string, error)
2020-12-27 09:05:24 +08:00
}
```
2020-12-29 18:02:44 +08:00
* *DescribeService*
2020-12-27 09:05:24 +08:00
```go
type QueryNodeDescription struct {
ResourceCost ResourceCost
}
type CollectionDescription struct {
ParitionIDs []UniqueID
}
type DbDescription struct {
CollectionDescriptions []CollectionDescription
}
2020-12-29 18:02:44 +08:00
type ServiceDescription struct {
2020-12-27 09:05:24 +08:00
DbDescriptions map[UniqueID]DbDescription
NodeDescriptions map[UniqueID]QueryNodeDescription
}
```
2020-12-29 18:02:44 +08:00
* *DescribeParition*
2020-12-27 09:05:24 +08:00
```go
2020-12-29 18:02:44 +08:00
type DescribeParitionRequest struct {
DbID UniqueID
CollectionID UniqueID
partitionIDs []UniqueID
}
2020-12-27 09:05:24 +08:00
type PartitionState = int
const (
NOT_EXIST PartitionState = 0
2021-01-08 17:20:59 +08:00
ON_DISK PartitionState = 1
PARTIAL_IN_MEMORY PartitionState = 2
IN_MEMORY PartitionState = 3
PARTIAL_IN_GPU PartitionState = 4
IN_GPU PartitionState = 5
2020-12-27 09:05:24 +08:00
)
2020-12-29 18:02:44 +08:00
type ResourceCost struct {
MemUsage int64
CpuUsage float32
}
2020-12-27 09:05:24 +08:00
type PartitionDescription struct {
ID UniqueID
State PartitionState
ResourceCost ResourceCost
}
2020-12-29 18:02:44 +08:00
type PartitionDescriptions struct {
PartitionDescriptions []PartitionDescription
}
```
* *CreateQueryChannel*
```go
type QueryChannels struct {
RequestChannel string
ResultChannel string
}
```
* *LoadPartitions*
```go
type LoadPartitonRequest struct {
DbID UniqueID
CollectionID UniqueID
PartitionIDs []UniqueID
}
```
* *ReleasePartitions*
```go
type ReleasePartitionRequest struct {
DbID UniqueID
CollectionID UniqueID
PartitionIDs []UniqueID
}
2020-12-27 09:05:24 +08:00
```
2021-01-04 14:16:43 +08:00
#### 8.2 Query Node
```go
type QueryNode interface {
Start() error
Close() error
2021-01-08 17:20:59 +08:00
AddQueryStream(requestStream MsgStream, resultStream MsgStream) error
RemoveQueryStream(requestStreamID string) error
WatchDmStreams(insertStreams MsgStream) error
WatchDdStream(stream MsgStream) error
SetTimeTickStream(stream MsgStream) error
SetStatsStream(stream MsgStream) error
2021-01-04 14:16:43 +08:00
2021-01-08 17:20:59 +08:00
LoadSegments(DbID UniqueID, CollID UniqueID, PartitionID UniqueID, SegIDs []UniqueID, FieldIDs []int64) error
ReleaseSegments(DbID UniqueID, CollID UniqueID, PartitionID UniqueID, SegIDs []UniqueID) error
DescribeParition(DbID UniqueID, CollID UniqueID, PartitionID UniqueID) (PartitionDescription, error)
2021-01-04 14:16:43 +08:00
}
```
2021-01-08 17:20:59 +08:00
2020-11-19 10:59:10 +08:00
#### 8.2 Collection Replica
$collectionReplica$ contains a in-memory local copy of persistent collections. In common cases, the system has multiple query nodes. Data of a collection will be distributed across all the available query nodes, and each query node's $collectionReplica$ will maintain its own share (only part of the collection).
Every replica tracks a value called tSafe which is the maximum timestamp that the replica is up-to-date.
###### 8.1.1 Collection
``` go
type Collection struct {
Name string
Id uint64
Fields map[string]FieldMeta
SegmentsId []uint64
cCollectionSchema C.CCollectionSchema
}
```
###### 8.1.2 Field Meta
```go
type FieldMeta struct {
Name string
Id uint64
IsPrimaryKey bool
TypeParams map[string]string
IndexParams map[string]string
}
```
###### 8.1.3 Segment
``` go
type Segment struct {
Id uint64
ParitionName string
CollectionId uint64
OpenTime Timestamp
CloseTime Timestamp
NumRows uint64
cSegment C.CSegmentBase
}
```
#### 8.3 Data Manipulation Service
```go
type manipulationService struct {
ctx context.Context
pulsarURL string
fg *flowgraph.TimeTickedFlowGraph
msgStream *msgstream.PulsarMsgStream
node *QueryNode
}
```