Are benchmarks useful for anything more than bragging rights? It’s a question we could debate for hours, but Qualcomm just released their new Vellamo 2.0 benchmark suite and we decided to take it for a spin with a handful of Android devices to see if we could form any drive-by conclusions. Read on to see seven Android devices go head-to-head and find out who comes out on top.
Grab Vellamo Mobile Benchmark 2.0 from the download link below. If you want to compare your results, make sure you go into the Vellamo settings and set Repeat Benchmarks to average five runs.
We ran Vellamo 2.0 on a variety of Android smartphones to see what the results could tell us. I chose the latest flagship devices from HTC, LG, and Samsung, and threw in the Nexus 7 tablet just for fun.
- AT&T HTC One X (Snapdragon S4 Plus, 4.0.3)
- Unlocked HTC One X (Tegra 3, Android 4.0.4)
- T-Mobile Samsung Galaxy S III (Snapdragon S4 Plus, Android )
- Unlocked LG Optimus 4X HD (Tegra 3, Android 4.0.3)
- Verizon Galaxy Nexus (OMAP4, Android 4.0.4)
- Unlocked Galaxy Nexus (OMAP4, Android 4.1.1)
- Nexus 7 (Tegra 3, Android 4.1.1)
Each device was rebooted before I started the benchmark and then I let it run five times to get the average score.
Metal Chapter Tests: The Metal Chapter is a set of discrete lower-level benchmarks around the CPU. This series of tests evaluates capabilities such as: CPU performance on integer and floating point operations, memory read/write and peak bandwidth performance, and memory branching speed.
Verizon Galaxy Nexus vs Unlocked Galaxy Nexus: These two devices feature nearly identical hardware, but one is running Android 4.0.4 and the other has the latest Android 4.1.1. We can clearly see that the upgrade to Jelly Bean has improved the browsing performance. The CPU performance appears to be mostly unchanged.
AT&T HTC One X vs Unlocked HTC One X: Here we have two devices with similar software, but two different processors. Vellamo appears to favor the Snapdragon S4 Plus in the Metal CPU tests, but the One X with Tegra achieved the highest result on the HTML5 browsing test.
Nexus 7 vs smartphones: In day-to-day use, the Nexus 7 feels like it has the fastest overall system performance. But in the Metal CPU test it came in last place. Clearly Google has optimized Jelly Bean to run great on the Nexus 7.
Snapdragon S4 vs Tegra 3: The HTML5 test appears to be pretty even, but the Metal CPU test clearly favors the Snapdragon S4. I don’t think any of the CPU tests are multi-threaded, or else we would see the Nexus 7 get a higher score above the aging Galaxy Nexus. Keep in mind that Qualcomm does produce the Vellamo benchmark, so some things could be optimized in their favor.
Vellamo 2.0: This is definitely one of the nicest looking benchmark suites I have used. I like that it runs a total of 21 tests and gives me a composite score. There are plenty of extra features included for testers like the ability to repeat tests multiple times and email detailed results.
Vellamo appears to be a useful tool for measuring the performance between two software builds on the same device. But when it comes to comparing multiple devices, I’m not sure what to make of it yet. Someone who just looks at the results of the Metal CPU test might think the Nexus 7 is the slowest, but that device actually has some of the fastest performance we have seen.
The benchmark is free for anyone to download, so give it a try and let us know how your device compares in the comments below.