• PocketSphinxQuantization

Acoustic Model Quantization in PocketSphinx

Data path

We need to determine how many significant bits are necessary at each step here. Currently, using fast SCHMM computation in fixed-point mode (which I think should actually just become the default for PocketSphinx), we have:

1. Acoustic data - signed 16-bit
2. FFT output - Q15 or Q31
3. Log power spectrum - Q19.12
4. MFCC - Q19.12
5. Log Euclidean distance - Q19.12
6. Log Mahalanobis distance - signed 32-bit
7. Gaussian densities - signed 32-bit
8. Normalized Gaussian densities - unsigned 8-bit (actually range of 0-96 !)
9. Acoustic scores - signed 32-bit (but always negative)
10. Language model scores - signed 32-bit (but also always negative)
11. Viterbi path scores - signed 32-bit (but also always negative)

Clearly it is a little bit silly to be using 32 bits for things everywhere when there is a "bottleneck" at the Gaussian mixture computation that is going to throw away all but 6.5 bits of information. The question is whether we need those extra bits for some of the intermediate values in order to avoid cascading errors.

For the FFT, I'm pretty sure that Q15 is sufficient. When we go to the log power spectrum and MFCC (which have the same dynamic range, albeit a different one from the FFT), we can definitely get away with 16 bits, it's just a question of how we use them. This is the right place to introduce some form of simple linear quantization. The maximum range of these values (based on C0) can actually be computed ahead of time based on the FFT size, but it is also reasonable to compute it based on the global mean and variance of the acoustic model.

The "baseline" in the future, without any kind of sophisticated quantization, should look more like this:

1. Acoustic data - signed 16-bit
2. FFT output - Q15
3. Log power spectrum - 16 bits
4. MFCC - 16 bits, linear quantization
5. Log Euclidean distance - 8 bits?
6. Log Mahalanobis distance - 8 bits?
7. Gaussian densities - 8 bits?
8. Normalized Gaussian densities - unsigned 8-bit (actually range of 0-96 !)
9. Acoustic scores - unsigned 16-bit (negated log-probs)
10. Language model scores - unsigned 16-bit (negated log-probs)
11. Viterbi path scores - unsigned 16-bit (negated log-probs)

Say that the dynamic range of the acoustic data is D bits, and the FFT order is M. Then the FFT output will have dynamic range of M+D bits, and the log of the FFT output will have a dynamic range of log(M+D) bits (but still should require D bits of precision). To obtain the power spectrum we square the absolute value of the DFT, giving us a dynamic range of log(M+D) + 1 bits.

References

Nokia evaluation of different model compression techniques

SDCHMM papers (original, training)

qHMM papers (original, adaptation)

PocketSUMMIT paper