Not Just Samsung: Almost All Companies Inflate Benchmarks

It was recently discovered that Samsung tampered with benchmark results for its new Galaxy Note 3 phablet-style smartphone, and while all the fingers are pointed at Samsung, it turns that almost all phone manufacturers are doing the same thing, according to a report by AnandTech.

The report provides evidence that suggests nearly all phone manufacturers, with the exception of Apple and Motorola, use CPU and GPU optimizations to make benchmark scores better than they truly are. Companies like LG, HTC and Asus are guilty of this as well, according to the report.

Essentially what happens is that companies include a variety of methods to enhance device performance when a benchmark app is detected to be running. During normal usage, the phone is clocked at normal speeds with default settings and everything, but once a benchmark app is opened, the phone detects that and goes into an ultra high performance mode of sorts, where limitations are lifted and CPU voltages and speeds are set to their highest setting.

galaxy-note-3

Something like this was detected on Samsung’s latest Galaxy Note 3, as well as its Galaxy S4 flagship smartphone that the company released earlier this year.

AnandTech put together a chart determining which devices are tampered with and which benchmarking apps the phones are tampered for, and turns out that almost every phone manufacturer has at least one phone that is guilty of inflating benchmark results, except for Apple and Motorola.

Read: Galaxy Note 3 Review: Samsung’s Most Powerful Phone

Advertisement

The only solution? There really isn’t one, but a way to make sure that you’re getting accurate benchmarking results is to use less popular benchmarking apps. As AnandTech notes, phone companies are altering their phones to work well with mostly just the popular benchmarking apps. It’s also noted that it’s important “to continue to evolve the suite ahead of those optimizing for it. The more attention you draw to certain benchmarks, the more likely they are to be gamed.”

It’s also important to note that benchmarks aren’t everything, and they actually have very little to do with the actual usage of the phone. Real-life usage of a smartphone doesn’t rely on benchmarks, and frankly we think companies are wasting their time trying to inflate them, but the sad truth is, spec nerds will always rely on benchmarks to decide what their next phone will be, and the faster benchmarks can boost sales for a company.

  

Comments

  1. Dustin says

    Yea, I try to overclock both CPU and GPU when benchmarking. Its not cheating if the device is capable of doing it (although the user should be in control when this happens, say a dialog should come up asking the user if it would be ok to go into turbo mode or something like that). If you want to see how fast your new lambo goes but it has a speed limiter set to 180, what’s the poin? All they are doing is temporarily pressing the track mode button and letting you see what its capable of.

  2. drakes says

    Samsung may be unlucky one or the one which does anything to score better benchmarks. Bad thing is that Samsung is the main target for bench reviewer now as it has already been accused of doing so in past. I am also of the feeling that all the companies does so for the sake of better benchmarks, but the good thing is that they did not get caught till now.

Leave a Reply