DXOMARK Now Include an Audio Score for Smartphones
Smartphone filmmakers, vloggers and MoJo-ers spend a lot of time exploring the visual capabilities of various smartphones. At the same time, DxOMark have made their name providing the most rigorous testing of smartphone cameras. However, it is a smartphone’s recording and playback of audio often gets left out of people’s thinking.
When I first started looking into using a smartphone for filmmaking, I had a Samsung S8 at the time. What really impressed me was the quality of the stereo audio recorded by the phone’s inbuilt mics.
While we all want a good mic, a smartphone’s audio recording capability is especially useful for mobile journalism and documentary shooting. In addition, anyone recording audio in a live situation might find themselves falling back on the inbuilt mics once in a while.
Like with cameras, there’s the physical capture device (lens or microphone). But there’s also processing going on which can make a big difference. DxOMark have provided an audio test of 4 of the top smartphone’s on the market:
iPhone and Samsung top
As you can hear from this test, the iPhone 11 Pro Max and the Samsung Galaxy S10+ come out on top. But it’s the OnePlus 7 Pro’s compression which ruins it’s audio, not the mic. Meanwhile, the Honor 20 Pro inbuilt microphone doesn’t have the tonal range (or has software removing the bass end).
Features Aimed at Consumers
Smartphone makers aim mostly at casual consumers (which makes the common “pro” tag rather ironic). For this reason, people wanting to use them in professional situations often find the quality compromised. Extra compression of the audio or the addition of a high pass filter is a good example.
Audio compression is useful for recording the spoken voice. Unfortunately, it also raises the level of the background ambience. And heavy compression becomes noticeable when recording live music, as it can be heard changing levels constantly.
A high pass filter (the high frequencies are allowed to “pass” the filter) is great for removing or reducing rumbling wind noises and breath pops. But it also reduces the breadth of the frequency, removing the bass end and resulting in a harsher sound.
While camera apps are great at overriding a smartphone’s camera software to provide manual control, they don’t do too much for audio. Perhaps a setting in FilmicPro to allow toggling between high pass filter on and off would be useful. And compression on and off too. That said, I don’t know if this is possible.
Google Pixel 4 Audio Test
DxOMark recently tested the audio capabilities of the new Google Pixel 4 smartphone. For an audio test, each phone is judged for Playback and Recording.
Playback is judged in 5 categories: Timbre, Dynamics, Spacial, Volume & Artifacts.
Recording is judged in 6 categories: Timbre, Dynamics, Spacial, Volume, Artifacts and Background.
The Pixel 4 comes out with an overall score of 68, which puts the phone in the top 5 of those tested.
While most of us smartphone filmmakers are going to be interested in the recording score, I guess we’ll be less interested in how a phone performs when playing audio. Therefore, the Pixel 4’s recording rating of 63 is pretty much the only one which really concerns us.
How relevant are the scores?
When it comes to audio, I have a good amount of knowledge, having been an audio engineer for about 20 years. And, to be honest, I would question sentences like, “Smartphones often rely on their small microphones for everything, and suppressing background noise relies on the device’s audio processing.”
As far I know, you can’t suppress background noise with software. Also (correct me if I’m wrong), but the first half of the sentence doesn’t seem to relate to the 2nd half. I mean, what does the size of the microphone have to do with suppressing background noise?
Compression will always increase background noise levels
Lavalier mics, for example, are small but good at reducing background noise (mostly due to placement). And a $10 lavalier will provide less audio quality than a $500 one. However, the only methods to “suppress background noise” that I know of involve microphone placement, using directional mics (like shotgun mics) or using soundproofing and damping equipment in a room.
There are no directional microphones contained within a smartphone. Therefore, there’s no way one smartphone can be better at reducing background noise than any other. Which leads me to question why they include it as a rating category.
One thing that does make background noise worse is compression. This is because it increases quieter sounds and decreases louder sounds (evening them out). Therefore, background noise will be boosted by adding compression. In other words, a smartphone can only make background noise worse, and that’s by adding compression.
Microphone specifications
Microphones are usually analysed using certain specs. Things like “impedance” “signal-to-noise ratio” and so on. The frequency response of a microphone would be indicated using a term such as “50 Hz to 15kHz”. But DxOMark testers don’t use such figures.
Instead, like with all their ratings, they simply state a number (from 1-100? based on…?). They talk about “a Dynamics sub-score of 67, the Pixel 4 achieves a good overall result, but falls behind Huawei Mate 20X’s 75”. But what does that actually mean? Not much to a sound engineer.
Then we get phrases like “The Pixel 4’s attack was sharp and maintained percussion fidelity, but became distorted as volume increased to its maximum level.” Then they provide very brief explanations for their terms (like attack and maximum level). Do they mean the compression used on the Pixel 4’s mic was sharp? If so, this would be useful in some situations and not in others – that’s why compressors come with the ability to adjust the attack setting.
You know, sometimes I get the feeling DxOMark are trying to sound scientific without too much actual science behind their review.
General Purpose Microphones
The microphones in our smartphones have to cover a range of uses, from dictaphone to recording live bands. A professional sound engineer would take a dim view of having to use the same mic for recording a single speaking voice as for recording 5 musicians. So exactly what are they trying to achieve with their audio reviews?
Do DxOMark aim their ratings at professionals? Apparently not. It seems they aim at a consumer who knows no better, and add a heavy dose of scientific sounding words and sentence structure.
What I would say is DxOMark reviews are fine. It’s good they’ve created a rating system so it’s easier to compare one phone against another without leaving their site. But they aren’t really any more scientific than other reviews, even though they sound it.
They are good and thorough reviews, albeit dressed up as experts in white lab coats testing smartphones scientifically. That’s not to say they have no value, but I don’t think they carry any greater value than a YouTube review or something written for any other online news source.
The audio test above is useful, especially of you want to record live music. If you’re interested in recording a solo speaking voice, then it doesn’t tell you too much. Also, like with other YouTube reviews, you are limited to the conditions the reviewer chooses for the test.
But what do you think?
Eager to learn more?
Join our weekly newsletter featuring inspiring stories, no-budget filmmaking tips and comprehensive equipment reviews to help you turn your film projects into reality!
