EEMBC sees the benchmark as having value to processor, system, and software vendors, but by having an easy-to-run tool for end-users to validate and compare operations on their phones or tablets, many of which vary considerably in performance, it opens up standardized benchmarking to a wider community.
Ultimately, AndEBench will comprise a battery of performance tests for mobile devices, but its initial focus is on the CPU and Dalvik interpreter performance. AndEBench 1.0 compares the Android platform’s native and Java performance. This will give users a quantified measure of the Java interpreters’ efficiency on a given platform and help them understand what performance gains are feasible.
AndEBench also tests a device’s multicore and/or multithreading capabilities. The user can change the number of executable threads to allow a comparison between single and multiple core operation.
“We developed AndEBench with a strong focus on showcasing the under-the-hood device behavior – and although AndEBench is not a fancy benchmark, it analyzes the processor’s capability as well as that of the Dalvik
interpreter’s efficiency,” said Shay Gal-On, EEMBC’s director of technology who coordinated the working group that produced the benchmark. “Furthermore, the collaborative effort of our working group members has ensured that AndEBench provides an equitable, unbiased, and repeatable test for mobile devices –critical to deliver data that can be used by technology providers and customers to fairly assess device performance.”
“Other Android benchmarks available on the market are ‘mom-and-pop’ benchmarks in which the motivation of the benchmark developer is unknown and results can vary significantly. Furthermore, since source code is typically unavailable, it is nearly impossible to verify the functionality or true usefulness of these benchmarks,” said Markus
Levy, EEMBC’s president. “EEMBC provides an industry-approved benchmark where the source code is available to members and licensees, providing a full understanding of the code internals, to encourage hardware and software developers to use the results to drive product improvements that matter to end-users.”