The most recent controversy in the Android world is Samsung’s cheating in benchmarks. This behavior was reported on the Galaxy S4, and reports are now showing that the Note 3 has the exact same behavior. Depending on the model, the device will ramp up CPU and even the GPU to get better scores in benchmarks, just to appear better than the competition.
The Snapdragon 800 variant of the Note 3 will ramp the CPU up to 100% load to gain a higher score than it would get if it gradually ramped up the CPU. The Exynos powered model will also boost the GPU frequency (by removing thermal throttling) to get better scores in many benchmarks. These apps are detected by name and the behavior is enabled, but not for all benchmarks.
People are raising a stink about Samsung doing all this, but many OEMs actually do this. Samsung is definitely not alone in this behavior; only current Motorola and Nexus devices are immune to it. So if you’re going to raise a stink, do it about everyone!
As you can see, many devices have some form of benchmark detection and cheating. Whether it be Snapdragon, Exynos or even Intel chips inside, it’s the manufacturer that implements it into software. Simply renaming the benchmark will actually stop it from optimizing, and you’ll see anywhere from a 3% to a 10% drop in performance for many of these devices.
Obviously, this is a bit of a dirty practice. Inflating benchmark scores provides unrealistic measurements of devices and breaks proper comparisons. All of these manufacturers should really remove any boosting software implemented in their devices, but we can’t see that happening anytime soon. Hopefully it doesn’t get more complicated, because the harder it is to get around, the more resources the company put into that software. And the less work it put into the actual device.
However, as customers, we can really make a difference. Write to these companies and demand that they stop this practice. Write to Google and ask them to add a requirement to their Play services certification. Write to your local government offi….. never mind. Hopefully we can get this practice to end soon enough.