The idea is to make the serialized message strongly typed, so the receiving program can deserialize the message without a template or recipe how to do that. It also forms an extra check on the correctness, and correspondence between the expected fields and the received fields. Several choices exist: defining message types using separate messages, preceding each message with its structure, or preceding each field with its structure. The current version of DJUTILS-SERIALIZATION implements the last case: every field is preceded by a byte that indicates the type of the byte(s) that follow.

When coding multi-byte values, endianness is of the utmost importance. Endianness indicates whether the most-significant byte comes first or whether the least-significant byte is sent first. The Internet and languages like Java use big endian, aka network byte order; the most significant byte comes first. Microsoft products and Intel processors internally use little endian; the least significant byte comes first. As an example, when we code an int (4 bytes) with the value 824, it is coded as follows using decimal notation (824 = 0 * 256^{3} + 0 * 256^{2} + 3 * 256 + 56 * 1):

`| 0 | 0 | 3 | 56 |`

Big endian`| 56 | 3 | 0 | 0 |`

Little endian

The following types have been implemented in the v1-version of the standard:

code | name | description |
---|---|---|

0 | BYTE_8 | Byte, 8 bit signed two's complement integer |

1 | SHORT_16 | Short, 16 bit signed two's complement integer |

2 | INT_32 | Integer, 32 bit signed two's complement integer |

3 | LONG_64 | Long, 64 bit signed two's complement integer |

4 | FLOAT_32 | Float, single-precision 32-bit IEEE 754 floating point |

5 | DOUBLE_64 | Float, double-precision 64-bit IEEE 754 floating point |

6 | BOOLEAN_8 | Boolean, sent / received as a byte; 0 = false, 1 = true |

7 | CHAR_8 | Char, 8-bit ASCII character |

8 | CHAR_16 | Char, 16-bit Unicode character, order for the 2 parts |

9 | STRING_8 | String, 32-bit number-preceded byte array of 8-bits characters |

10 | STRING_16 | String, 32-bit number-preceded char array of 16-bits characters, each 2-byte character in the order of the endianness |

11 | BYTE_8_ARRAY | Byte array, preceded by a 32-bit number indicating the number of bytes |

12 | SHORT_16_ARRAY | Short array, preceded by a 32-bit number indicating the number of shorts |

13 | INT_32_ARRAY | Integer array, preceded by a 32-bit number indicating the number of integers |

14 | LONG_64_ARRAY | Long array, preceded by a 32-bit number indicating the number of longs |

15 | FLOAT_32_ARRAY | Float array, preceded by a 32-bit number indicating the number of floats |

16 | DOUBLE_64_ARRAY | Double array, preceded by a 32-bit number indicating the number of doubles |

17 | BOOLEAN_8_ARRAY | Boolean array, preceded by a 32-bit number indicating the number of booleans |

18 | BYTE_8_MATRIX | Byte matrix, preceded by a 32-bit number row count and a 32-bit number column count |

19 | SHORT_16_MATRIX | Short matrix, preceded by a 32-bit number row count and a 32-bit number column count |

20 | INT_32_MATRIX | Integer matrix, preceded by a 32-bit number row count and a 32-bit number column count |

21 | LONG_64_MATRIX | Long matrix, preceded by a 32-bit number row count and a 32-bit number column count |

22 | FLOAT_32_MATRIX | Float matrix, preceded by a 32-bit number row count and a 32-bit number column count |

23 | DOUBLE_64_MATRIX | Double matrix, preceded by a 32-bit number row count and a 32-bit number column count |

24 | BOOLEAN_8_MATRIX | Boolean matrix, preceded by a 32-bit number row count and a 32-bit number column count |

25 | FLOAT_32_UNIT | Float stored internally as a float in the corresponding SI unit, with unit type and display unit attached. The total size of the object is 7 bytes |

26 | DOUBLE_64_UNIT | Double stored internally as a double in the corresponding SI unit, with unit type and display unit attached. The total size of the object is 11 bytes |

27 | FLOAT_32_UNIT_ARRAY | Dense float array, preceded by a 32-bit number indicating the number of floats, with unit type and display unit attached to the entire float array |

28 | DOUBLE_64_UNIT_ARRAY | Dense double array, preceded by a 32-bit number indicating the number of doubles, order, with unit type and display unit attached to the entire double array |

29 | FLOAT_32_UNIT_MATRIX | Dense float matrix, preceded by a 32-bit row count int and a 32-bit column count int, with unit type and display unit attached to the entire float matrix |

30 | DOUBLE_64_UNIT_MATRIX | Dense double matrix, preceded by a 32-bit row count int and a 32-bit column count int, with unit type and display unit attached to the entire double matrix |

31 | FLOAT_32_UNIT2_MATRIX | Dense float matrix, preceded by a 32-bit row count int and a 32-bit column count int, with a unique unit type and display unit per row of the float matrix. |

32 | DOUBLE_64_UNIT2_MATRIX | Dense double matrix, preceded by a 32-bit row count int and a 32-bit column count int, with a unique unit type and display unit per row of the double matrix. |

Unicode characters can be of different formats: UTF-8, UTF-16 with a byte-order marker (BOM), UTF-16BE (big-endian), UTF-16LE (little-endian), UTF-32 with a byte-order marker (BOM), UTF-32BE (big-endian), and UTF-32LE (little-endian). To code all Unicode characters in UTF-8, one to four UTF-8 bytes are needed through the use of escape characters. For UTF-16, one or two two-byte combinations are needed. In UTF-32, all Unicode characters can be directly coded. Because of the escape characters, characters and strings really look different in UTF-8, UTF-16, and UTF-32. The curent version of DJUTILS-SERIALIZATION (and Sim0MQ) support UTF-8, UTF-16BE (big-endian), and UTF-16LE (little-endian). Mode about the differences between the encodings is explained in the Unicode FAQ list: https://unicode.org/faq/utf_bom.html#gen6.

For a discussion on little and big endianness for UTF-8 and UTF-16 strings, see the following discussion at StackExchange: https://stackoverflow.com/questions/3833693/isn-t-on-big-endian-machines-utf-8s-byte-order-different-than-on-little-endian, as well as https://unicode.org/faq/utf_bom.html#utf8-2.

Note that because of escape characters (or surrogates), the String length is not equal to the number of bytes in UTF-8, nor to the number of bytes divided by two in UTF-16. The numbers in STRING_8, STRING_16, STRING_8_LE and STRING_16_LE are related to the number of bytes / shorts in the representation, and **not** to the number of characters in the resulting String.