PESQ: How to Predict MOS Using Mathematics

Therefore, we turn to how the predictions of voice quality can actually be made electronically. ITU P. 862 introduces Perceptual Evaluation of Speech Quality, the PESQ metric. PESQ is designed to take into account all aspects of voice quality, from the distortion of the codecs themselves to the effects of filtering, delay variation, and dropouts or strange distortions. PESQ was verified with a number of real MOS experiments to make sure that the numbers are reasonable within the range of normal telephone voices.

PESQ is measured on a 1 to 4.5 scale, aligning exactly with the 1 to 5 MOS scale, in the sense that a 1 is a 1, a 2 is a 2, and so on. (The area from 4.5 to 5 in PESQ is not addressed.) PESQ is designed to take into account many different factors that alter the perception of the quality of voice.

The basic concept of PESQ is to have a piece of software or test measurement equipment compare two versions of a recording: the original one and the one distorted by the telephone equipment being measured. PESQ then returns with the expected mean opinion score a group of real listeners are likely to have thought.

PESQ uses a perceptual model of voice, much the same way as perceptual voice codecs do. The two audio samples are mapped and remapped, until they take into account known perceptual qualities, such as the human change in sensitivity to loudness over frequency (sounds get quieter at the same pressure levels as they get higher in pitch). The samples are then matched up in time, eliminating any absolute delay, which affects the quality of a phone call but not a recording. The speech is then broken up into chunks, called utterances, which correspond to the same sound in both the original and distorted recording. The delays and distortions are then analyzed, counted, and correlated, and a number measuring how far removed the distorted signal is from the original signal is presented. This is the PESQ score.

PESQ is our first entry into the area of mathematical, or algorithmic, determination of call quality. It is good for measuring how well a new codec works, or how much noise is being injected into the sample. However, because it requires comparing what the talker said and what the listener heard, it is not practical for real-time call quality measurements.

Mean Opinion Score and How it Sounds

The Mean Opinion Score, or MOS (sometimes redundantly called the MOS score), is one way of ranking the quality of a phone call. This score is set on a five-point scale, according to the following ranking:

• 5. Excellent
• 4. Good
• 3. Fair
• 2. Poor
• 1. Bad

MOS never goes below 1, or above 5.

There is quite a science to establishing how to measure MOS based on real-world human studies, and the depth they go into is astounding. ITU P.800 lays out procedures for measuring MOS. Annex B of P.800 defines listening tests to determine quality in an absolute manner. The test requirements are spelt out in detail. The room to be used should be between 30 and 120 cubic meters, to ensure the echo remains within known values. The phone under test is used to record a series of phrases. The listeners are brought in, having been selected from a group that has never heard the recorded sentence lists, in order to avoid bias. The listeners are asked to mark the quality of the played-back speech, distorted as it may be by the phone system. The listeners' scores, on the one-to-five scale, are averaged, and this becomes the MOS for the system. The goal of all of this is to attempt to increase the repeatability of such experiments.

Clearly, performing MOS tests is not something that one would imagine can be done for most voice mobility networks. However, the MOS scale is so well known that the 1 to 5 scale is used as the standard yardstick for all voice quality metrics. The most important rule of thumb for the MOS scale is this: a MOS of 4.0 or better is toll-quality. This is the quality that voice mobility networks have to achieve, because this is the quality that nonmobility voice networks provide every day. Forgiveness will likely offered by users when the problem is well known and entirely relatable, such as for bad-quality calls when in a poor cellular coverage area. But, once inside the building, enterprise voice mobility users expect the same quality wirelessly as they do when using their desk phone.

Thus, when a device reports the MOS for a call, the number you are seeing has been generated electronically, based on formulas that are thought to be reasonable facsimiles of the human experience.