This is a benchmark application for the iPhone or the iPod touch. Why is this interesting?

• You would like to check performance of your device after some changes or after installing some apps.

• You own different devices and would like to understand their performance differences.

• You are a developer and would like to learn about performance differences between the devices and the sim.

• You are just curious whether you have a standard model or someone in the fab but it the wrong chips, accidentally.

• You just like benchmarking your hardware!

Features:

• Testing integer and floating point calculation performance by performing 100thousands of operations on randomly generated numbers.

• Tesing 2D graphics drawing performance by drawing a CoreGraphics drawing area.

• Testing memory allocation performance by randomly generating strings (about 1MB) and releasing them.

• Testing file writing performance by writing randomly generated strings to a file (about 1MB).

• Testing download rate performance by downloading random data from a Google App Engine App at 1, 4, 32, and 128 kB sizes.

I have written this app, because I have experienced with my first application "ChartView" the following: there is a surprisingly high performance difference between the iPhone simulator and the real iPhone. So, I got curious about the iPhone's performance differences and I wrote some tests.

Moreover, I have read in summer 2009 about the introduction of the 3GS. I was surprised that a magazine has compared the 3GS by measuring some tasks with their watches. Consequently, a benchmark program in the good old style of Speedometer is a nice tool (anyone remembers this app for System 6?). Of course, this app is much simpler and less detailed than Speedometer. It is an iPhone app that covers memory allocation, floating point calculations, writing dictionaries and arrays into files and performs some drawing tasks.

Important Points

You might want to turn the airplane mode of your iPhone to on when running the benchmarks. The networking should be switched off for the benchmarks. Also, if your iPhone auto locks you out during running the tests, the results become really different.

If you got an idea for another benchmark, please let me know. I am really interested in extending the application with more benchmarks. As you might have noticed, I have provided a table structure which supports more than existing entries / tests.

I did not provide a overall score first. I have had my doubts about it, because the overall score would require a weighting. What weighting would b appropriate? A general weighting scheme appears difficult to determine. For example, some apps might not have any floating point calculations at all, while for other apps, it represents the most important part. Nevertheless, I like the idea of a general "BenchTest Score". Since this feature qas requested a number of times, I have added it.

Contact

Please send all comments, suggestions and ideas to:

Technical Details

The individual tests are not very complex. Every test is repeated for 50 times. Then, the durations of the individual runs are sorted and the last and first five results are skipped. The remaining 40 values are considered for a mean calculation that is also displayed to the user. By this way, exceptionally long or exceptionally short runs are omitted and values that occur at 80% of the time are taken. The goal of these calculations is to achieve repeatable results. So, in general, for every run at different times, the result should be quite the same.

Memory Test

The memory test randomly generates string values and put them in to arrays and dictionaries. The amount of randomly generated string data is about at 1MB. Afterwards the data is dealloc'd again.

Floating Point Test

The floating point test will generate 100.000 random floating point numbers and it will perform 32 calculations from the CoreServices lib. Operations include sin, cosin, asin, tan, atan, etc. Consider that the calculation of a sin value from a floating-point number is not exactly the same kind of operations that is covered by GFLOP or MFLOP benchmarks.

Integer Test

This test will generate 250.000 random int numbers and it will perform 2 add and 2 multiply operations per number. Please keep in mind that this bench does not control how the compiler arranges these operations for running them on the CPU. So the seconds taken for the 1 million operations is not equal to the MIPS performance of the processor.

Graphics Test

The graphics test is probably the most controversial one. As Speedometer users will remember, this app has had generated a lot of lines, rects and circles which the user actually saw displaying. With the iPhone this gets a little bit more difficult. In standard iPhone applications, the UI is drawn offscreen and then displayed. You can work with animations, but since you define the duration for an animation a real benchmark is difficult here. Also, you could manipulate the drawing procedure with timers. However, that looks like a strange patch that involves also other issues when doing measurements. How much real world would be in there then? Because the standard way of dawing is used in most iPhone apps by what is done in - (void)drawRect:(CGRect)rect, I have decided in favor of doing exactly the same for benchmarking. So, that is why drawing is offscreen.

Download Test

This test calls a Google App Engine App for generating random data at different sizes. This is to prevent any caching on the way like caching from a proxy server of your network or the network of your Internet provider. The results shown are the takes of three download attempts. I could do 10 downloads, of course. However, I feel uncertain about the potential traffic.

Credits

Well, I wrote the app, but I would like to thank Robert from tinb software for first feedback and advice.
Also, my thanks to Stephan M., Gelu, Martin V., Greg, Neuwalker, "GRIMM", "KRASS", Chris, Jaehyun, Martin R. and all others who submitted their results!

Legal Issues

THE SERVICES AND THE CONTENT OR THE BENCHTEST APPLICATION ARE PROVIDED ON AN "AS IS" BASIS, AND THE AUTHORS OF BENCHTEST EXPRESSLY DISCLAIMS ANY AND ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, WITH RESPECT TO THE SERVICES OR ANY CONTENT AND RESULTS.

IN NO EVENT SHALL THE AUTHORS OF BENCHTEST BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, PUNITIVE, OR CONSEQUENTIAL DAMAGES OF ANY KIND WHATSOEVER WITH RESPECT TO THE FUNCTIONALITY, THE CONTENT AND THE RESULTS.

This means that the authors of BENCHTEST will not be liable for any consequential, incidental or special damages or financial losses resulting from the information and calculations, use or inability to use BENCHTEST, or any errors or omissions in the use of BENCHTEST.

The authors of BENCHTEST have taken all reasonable efforts to ensure that the information and calculations by BENCHTEST are complete and accurate.

The authors of BENCHTEST reserve the right to update the functionality of the application, the license model or its pricing, without specific notice or warning to users. Functionality of benchmarking algorithms of BENCHTEST may change with application updates without prior notice to individual users.

The authors of BENCHTEST advise its users that when accessing an external Internet web site, the authors of BENCHTEST do not take any responsibility for the relevance and accuracy of its content.

All trademarks and registered trademarks are the property of their respective owners.