1 // Copyright 2009 The Go Authors. All rights reserved.
2 // Use of this source code is governed by a BSD-style
3 // license that can be found in the LICENSE file.
6 Package gob manages streams of gobs - binary values exchanged between an
7 Encoder (transmitter) and a Decoder (receiver). A typical use is transporting
8 arguments and results of remote procedure calls (RPCs) such as those provided by
11 A stream of gobs is self-describing. Each data item in the stream is preceded by
12 a specification of its type, expressed in terms of a small set of predefined
13 types. Pointers are not transmitted, but the things they point to are
14 transmitted; that is, the values are flattened. Recursive types work fine, but
15 recursive values (data with cycles) are problematic. This may change.
17 To use gobs, create an Encoder and present it with a series of data items as
18 values or addresses that can be dereferenced to values. The Encoder makes sure
19 all type information is sent before it is needed. At the receive side, a
20 Decoder retrieves values from the encoded stream and unpacks them into local
23 The source and destination values/types need not correspond exactly. For structs,
24 fields (identified by name) that are in the source but absent from the receiving
25 variable will be ignored. Fields that are in the receiving variable but missing
26 from the transmitted type or value will be ignored in the destination. If a field
27 with the same name is present in both, their types must be compatible. Both the
28 receiver and transmitter will do all necessary indirection and dereferencing to
29 convert between gobs and actual Go values. For instance, a gob type that is
34 can be sent from or received into any of these Go types:
36 struct { A, B int } // the same
37 *struct { A, B int } // extra indirection of the struct
38 struct { *A, **B int } // extra indirection of the fields
39 struct { A, B int64 } // different concrete value type; see below
41 It may also be received into any of these:
43 struct { A, B int } // the same
44 struct { B, A int } // ordering doesn't matter; matching is by name
45 struct { A, B, C int } // extra field (C) ignored
46 struct { B int } // missing field (A) ignored; data will be dropped
47 struct { B, C int } // missing field (A) ignored; extra field (C) ignored.
49 Attempting to receive into these types will draw a decode error:
51 struct { A int; B uint } // change of signedness for B
52 struct { A int; B float } // change of type for B
53 struct { } // no field names in common
54 struct { C, D int } // no field names in common
56 Integers are transmitted two ways: arbitrary precision signed integers or
57 arbitrary precision unsigned integers. There is no int8, int16 etc.
58 discrimination in the gob format; there are only signed and unsigned integers. As
59 described below, the transmitter sends the value in a variable-length encoding;
60 the receiver accepts the value and stores it in the destination variable.
61 Floating-point numbers are always sent using IEEE-754 64-bit precision (see
64 Signed integers may be received into any signed integer variable: int, int16, etc.;
65 unsigned integers may be received into any unsigned integer variable; and floating
66 point values may be received into any floating point variable. However,
67 the destination variable must be able to represent the value or the decode
70 Structs, arrays and slices are also supported. Strings and arrays of bytes are
71 supported with a special, efficient representation (see below).
73 Functions and channels cannot be sent in a gob. Attempting
74 to encode a value that contains one will fail.
76 The rest of this comment documents the encoding, details that are not important
77 for most users. Details are presented bottom-up.
79 An unsigned integer is sent one of two ways. If it is less than 128, it is sent
80 as a byte with that value. Otherwise it is sent as a minimal-length big-endian
81 (high byte first) byte stream holding the value, preceded by one byte holding the
82 byte count, negated. Thus 0 is transmitted as (00), 7 is transmitted as (07) and
83 256 is transmitted as (FE 01 00).
85 A boolean is encoded within an unsigned integer: 0 for false, 1 for true.
87 A signed integer, i, is encoded within an unsigned integer, u. Within u, bits 1
88 upward contain the value; bit 0 says whether they should be complemented upon
89 receipt. The encode algorithm looks like this:
93 u = (^i << 1) | 1 // complement i, bit 0 is 1
95 u = (i << 1) // do not complement i, bit 0 is 0
99 The low bit is therefore analogous to a sign bit, but making it the complement bit
100 instead guarantees that the largest negative integer is not a special case. For
101 example, -129=^128=(^256>>1) encodes as (FE 01 01).
103 Floating-point numbers are always sent as a representation of a float64 value.
104 That value is converted to a uint64 using math.Float64bits. The uint64 is then
105 byte-reversed and sent as a regular unsigned integer. The byte-reversal means the
106 exponent and high-precision part of the mantissa go first. Since the low bits are
107 often zero, this can save encoding bytes. For instance, 17.0 is encoded in only
108 three bytes (FE 31 40).
110 Strings and slices of bytes are sent as an unsigned count followed by that many
111 uninterpreted bytes of the value.
113 All other slices and arrays are sent as an unsigned count followed by that many
114 elements using the standard gob encoding for their type, recursively.
116 Maps are sent as an unsigned count followed by that man key, element
117 pairs. Empty but non-nil maps are sent, so if the sender has allocated
118 a map, the receiver will allocate a map even no elements are
121 Structs are sent as a sequence of (field number, field value) pairs. The field
122 value is sent using the standard gob encoding for its type, recursively. If a
123 field has the zero value for its type, it is omitted from the transmission. The
124 field number is defined by the type of the encoded struct: the first field of the
125 encoded type is field 0, the second is field 1, etc. When encoding a value, the
126 field numbers are delta encoded for efficiency and the fields are always sent in
127 order of increasing field number; the deltas are therefore unsigned. The
128 initialization for the delta encoding sets the field number to -1, so an unsigned
129 integer field 0 with value 7 is transmitted as unsigned delta = 1, unsigned value
130 = 7 or (01 07). Finally, after all the fields have been sent a terminating mark
131 denotes the end of the struct. That mark is a delta=0 value, which has
134 Interface types are not checked for compatibility; all interface types are
135 treated, for transmission, as members of a single "interface" type, analogous to
136 int or []byte - in effect they're all treated as interface{}. Interface values
137 are transmitted as a string identifying the concrete type being sent (a name
138 that must be pre-defined by calling Register), followed by a byte count of the
139 length of the following data (so the value can be skipped if it cannot be
140 stored), followed by the usual encoding of concrete (dynamic) value stored in
141 the interface value. (A nil interface value is identified by the empty string
142 and transmits no value.) Upon receipt, the decoder verifies that the unpacked
143 concrete item satisfies the interface of the receiving variable.
145 The representation of types is described below. When a type is defined on a given
146 connection between an Encoder and Decoder, it is assigned a signed integer type
147 id. When Encoder.Encode(v) is called, it makes sure there is an id assigned for
148 the type of v and all its elements and then it sends the pair (typeid, encoded-v)
149 where typeid is the type id of the encoded type of v and encoded-v is the gob
150 encoding of the value v.
152 To define a type, the encoder chooses an unused, positive type id and sends the
153 pair (-type id, encoded-type) where encoded-type is the gob encoding of a wireType
154 description, constructed from these types:
156 type wireType struct {
162 type ArrayType struct {
167 type CommonType struct {
168 Name string // the name of the struct type
169 Id int // the id of the type, repeated so it's inside the type
171 type SliceType struct {
175 type StructType struct {
177 Field []*fieldType // the fields of the struct.
179 type FieldType struct {
180 Name string // the name of the field.
181 Id int // the type id of the field, which must be already defined
183 type MapType struct {
189 If there are nested type ids, the types for all inner type ids must be defined
190 before the top-level type id is used to describe an encoded-v.
192 For simplicity in setup, the connection is defined to understand these types a
193 priori, as well as the basic gob types int, uint, etc. Their ids are:
203 // gap for reserved ids.
210 // 22 is slice of fieldType.
213 Finally, each message created by a call to Encode is preceded by an encoded
214 unsigned integer count of the number of bytes remaining in the message. After
215 the initial type name, interface values are wrapped the same way; in effect, the
216 interface value acts like a recursive invocation of Encode.
218 In summary, a gob stream looks like
220 (byteCount (-type id, encoding of a wireType)* (type id, encoding of a value))*
222 where * signifies zero or more repetitions and the type id of a value must
223 be predefined or be defined before the value in the stream.
230 Tokens starting with a lower case letter are terminals; int(n)
231 and uint(n) represent the signed/unsigned encodings of the value n.
236 uint(lengthOfMessage) Message
238 TypeSequence TypedValue
240 (TypeDefinition DelimitedTypeDefinition*)?
241 DelimitedTypeDefinition:
242 uint(lengthOfTypeDefinition) TypeDefinition
246 int(-typeId) encodingOfWireType
248 SingletonValue | StructValue
252 builtinValue | ArrayValue | MapValue | SliceValue | StructValue | InterfaceValue
254 NilInterfaceValue | NonNilInterfaceValue
257 NonNilInterfaceValue:
258 ConcreteTypeName TypeSequence InterfaceContents
260 uint(lengthOfName) [already read=n] name
262 int(concreteTypeId) DelimitedValue
266 uint(n) FieldValue*n [n elements]
268 uint(n) (FieldValue FieldValue)*n [n (key, value) pairs]
270 uint(n) FieldValue*n [n elements]
272 (uint(fieldDelta) FieldValue)*
276 For implementers and the curious, here is an encoded example. Given
277 type Point struct {X, Y int}
280 the bytes transmitted that encode p will be:
281 1f ff 81 03 01 01 05 50 6f 69 6e 74 01 ff 82 00
282 01 02 01 01 58 01 04 00 01 01 59 01 04 00 00 00
283 07 ff 82 01 2c 01 42 00
284 They are determined as follows.
286 Since this is the first transmission of type Point, the type descriptor
287 for Point itself must be sent before the value. This is the first type
288 we've sent on this Encoder, so it has type id 65 (0 through 64 are
291 1f // This item (a type descriptor) is 31 bytes long.
292 ff 81 // The negative of the id for the type we're defining, -65.
293 // This is one byte (indicated by FF = -1) followed by
294 // ^-65<<1 | 1. The low 1 bit signals to complement the
295 // rest upon receipt.
297 // Now we send a type descriptor, which is itself a struct (wireType).
298 // The type of wireType itself is known (it's built in, as is the type of
299 // all its components), so we just need to send a *value* of type wireType
300 // that represents type "Point".
301 // Here starts the encoding of that value.
302 // Set the field number implicitly to -1; this is done at the beginning
303 // of every struct, including nested structs.
304 03 // Add 3 to field number; now 2 (wireType.structType; this is a struct).
305 // structType starts with an embedded commonType, which appears
306 // as a regular structure here too.
307 01 // add 1 to field number (now 0); start of embedded commonType.
308 01 // add 1 to field number (now 0, the name of the type)
309 05 // string is (unsigned) 5 bytes long
310 50 6f 69 6e 74 // wireType.structType.commonType.name = "Point"
311 01 // add 1 to field number (now 1, the id of the type)
312 ff 82 // wireType.structType.commonType._id = 65
313 00 // end of embedded wiretype.structType.commonType struct
314 01 // add 1 to field number (now 1, the field array in wireType.structType)
315 02 // There are two fields in the type (len(structType.field))
316 01 // Start of first field structure; add 1 to get field number 0: field[0].name
318 58 // structType.field[0].name = "X"
319 01 // Add 1 to get field number 1: field[0].id
320 04 // structType.field[0].typeId is 2 (signed int).
321 00 // End of structType.field[0]; start structType.field[1]; set field number to -1.
322 01 // Add 1 to get field number 0: field[1].name
324 59 // structType.field[1].name = "Y"
325 01 // Add 1 to get field number 1: field[0].id
326 04 // struct.Type.field[1].typeId is 2 (signed int).
327 00 // End of structType.field[1]; end of structType.field.
328 00 // end of wireType.structType structure
329 00 // end of wireType structure
331 Now we can send the Point value. Again the field number resets to -1:
333 07 // this value is 7 bytes long
334 ff 82 // the type number, 65 (1 byte (-FF) followed by 65<<1)
335 01 // add one to field number, yielding field 0
336 2c // encoding of signed "22" (0x22 = 44 = 22<<1); Point.x = 22
337 01 // add one to field number, yielding field 1
338 42 // encoding of signed "33" (0x42 = 66 = 33<<1); Point.y = 33
339 00 // end of structure
341 The type encoding is long and fairly intricate but we send it only once.
342 If p is transmitted a second time, the type is already known so the
345 07 ff 82 01 2c 01 42 00
347 A single non-struct value at top level is transmitted like a field with
348 delta tag 0. For instance, a signed integer with value 3 presented as
349 the argument to Encode will emit:
355 03 // this value is 3 bytes long
356 04 // the type number, 2, represents an integer
OSDN Git Service
