Quantization is used to store floats using less space. An array of quantized data will be accompanied by a byte describing the quantization. The high nibble of the quantization byte tells you the data type, and the low nibble gives you a value to divide each value by.
format = quantization >> 4
divisor = 1 << (quantization & 0b1111)Supposedly, the data types are
| Format | Data type |
|---|---|
| 1 | float |
| 2 | u16 |
| 3 | s16 |
| 4 | u8 |
| 5 | s8 |
But in practice, I've run into formats 0, 3, and 4, and found that 0 and 3 are u16 and 4 is float.