test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 20:09:57 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity

[skip ci] Update doc of descriptor event format (#8487)

Browse Source 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
This commit is contained in:
dragondriver 2021-09-24 18:17:56 +08:00 committed by GitHub
parent 104882e102
commit 7a76022f3b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 16 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
docs/developer_guides/chap08_binlog.md
View File

@ -19,13 +19,11 @@ Binlog file consists of 4 bytes magic number and a series of events. The first e
| header +----------------------------+---------------------------------------------------------------------+ | header +----------------------------+---------------------------------------------------------------------+
| | TypeCode 8 : 1 | event type code | | | TypeCode 8 : 1 | event type code |
| +----------------------------+---------------------------------------------------------------------+ | +----------------------------+---------------------------------------------------------------------+
| | ServerID 9 : 4 | write node id | | | EventLength 9 : 4 | length of event, including header and data |
| +----------------------------+---------------------------------------------------------------------+ | +----------------------------+---------------------------------------------------------------------+
| | EventLength 13 : 4 | length of event, including header and data | | | NextPosition 13 : 4 | offset of next event from the start of file |
| +----------------------------+---------------------------------------------------------------------+
| | NextPosition 17 : 4 | offset of next event from the start of file |
+=====================================+=====================================================================+ +=====================================+=====================================================================+
| event | fixed part 21 : x | | | event | fixed part 17 : x | |
| data +----------------------------+---------------------------------------------------------------------+ | data +----------------------------+---------------------------------------------------------------------+
| | variable part | | | | variable part | |
+=====================================+=====================================================================+ +=====================================+=====================================================================+
@ -40,33 +38,29 @@ Binlog file consists of 4 bytes magic number and a series of events. The first e
| header +----------------------------+---------------------------------------------------------------------+ | header +----------------------------+---------------------------------------------------------------------+
| | TypeCode 8 : 1 | event type code | | | TypeCode 8 : 1 | event type code |
| +----------------------------+---------------------------------------------------------------------+ | +----------------------------+---------------------------------------------------------------------+
| | ServerID 9 : 4 | write node id | | | EventLength 9 : 4 | length of event, including header and data |
| +----------------------------+---------------------------------------------------------------------+ | +----------------------------+---------------------------------------------------------------------+
| | EventLength 13 : 4 | length of event, including header and data | | | NextPosition 13 : 4 | offset of next event from the start of file |
| +----------------------------+---------------------------------------------------------------------+
| | NextPosition 17 : 4 | offset of next event from the start of file |
+=====================================+=====================================================================+ +=====================================+=====================================================================+
| event | BinlogVersion 21 : 2 | binlog version | | event | CollectionID 17 : 8 | collection id |
| data +----------------------------+---------------------------------------------------------------------+ | data +----------------------------+---------------------------------------------------------------------+
| | ServerVersion 23 : 8 | write node version | | | PartitionID 25 : 8 | partition id (schema column does not need) |
| +----------------------------+---------------------------------------------------------------------+ | +----------------------------+---------------------------------------------------------------------+
| | CommitID 31 : 8 | commit id of the programe in git | | | SegmentID 33 : 8 | segment id (schema column does not need) |
| +----------------------------+---------------------------------------------------------------------+ | +----------------------------+---------------------------------------------------------------------+
| | HeaderLength 39 : 1 | header length of other event | | | FieldID 41 : 8 | field id (schema column does not need) |
| +----------------------------+---------------------------------------------------------------------+ | +----------------------------+---------------------------------------------------------------------+
| | CollectionID 40 : 8 | collection id | | | StartTimestamp 49 : 8 | minimum timestamp allocated by master of all events in this file |
| +----------------------------+---------------------------------------------------------------------+ | +----------------------------+---------------------------------------------------------------------+
| | PartitionID 48 : 8 | partition id (schema column does not need) | | | EndTimestamp 57 : 8 | maximum timestamp allocated by master of all events in this file |
| +----------------------------+---------------------------------------------------------------------+ | +----------------------------+---------------------------------------------------------------------+
| | SegmentID 56 : 8 | segment id (schema column does not need) | | | PayloadDataType 65 : 4 | data type of payload |
| +----------------------------+---------------------------------------------------------------------+ | +----------------------------+---------------------------------------------------------------------+
| | StartTimestamp 64 : 1 | minimum timestamp allocated by master of all events in this file | | | ExtraLength 69 : 4 | length of extra information |
| +----------------------------+---------------------------------------------------------------------+ | +----------------------------+---------------------------------------------------------------------+
| | EndTimestamp 65 : 1 | maximum timestamp allocated by master of all events in this file | | | ExtraBytes 73 : n | extra information in json format |
| +----------------------------+---------------------------------------------------------------------+ | +----------------------------+---------------------------------------------------------------------+
| | PayloadDataType 66 : 1 | data type of payload | | | PostHeaderLengths n : n | header lengths for all event types |
| +----------------------------+---------------------------------------------------------------------+
| | PostHeaderLength 67 : n | header lengths for all event types |
+=====================================+=====================================================================| +=====================================+=====================================================================|
``` ```

2
internal/storage/event_data.go
View File

@ -23,10 +23,10 @@ import (
type descriptorEventData struct { type descriptorEventData struct {
DescriptorEventDataFixPart DescriptorEventDataFixPart
PostHeaderLengths []uint8
ExtraLength int32 ExtraLength int32
ExtraBytes []byte ExtraBytes []byte
Extras map[string]interface{} Extras map[string]interface{}
PostHeaderLengths []uint8
} }
// DescriptorEventDataFixPart is a memorty struct saves events' DescriptorEventData. // DescriptorEventDataFixPart is a memorty struct saves events' DescriptorEventData.