Iteolih: Python Benchmark
24 May 2009 Markus Koetter iteolih python
As the plan is to embedd python as scripting language into the honeypot, I ran a benchmark on a testsuite. The ’testsuite’ is a c core which accepts connections, and allows python to deal with the input. The protocol used for benchmarking is http, the service serves a non static html page.
I tested
-
- 2.6.2_(release26-maint,_Apr_19_2009,_02:15:38)
-
- 3.0.1+_(r301:69556,_Apr_15_2009,_17:22:45)_
-
- 3.1a1+_(py3k,_Mar_30_2009,_02:02:26)_
To benchmark, I ran the apache benchmark tool ab
ab -n 100000 -c 15 http://localhost:8080/bar
The result:
| Server Software | 2.6.2 | 3.0.1 | 3.1a |
|---|---|---|---|
| Server Hostname | localhost | localhost | localhost |
| Server Port | 8080 | 8080 | 8080 |
| Document Path | /bar | /bar | /bar |
| Document Length | 191 bytes | 191 bytes | 191 bytes |
| Concurrency Level | 15 | 15 | 15 |
| Time taken for tests (seconds) | 28.47 | 102.15 | 31.8 |
| Complete requests | 100000 | 100000 | 100000 |
| Failed requests | 0 | 0 | 0 |
| Write errors | 0 | 0 | 0 |
| Total transferred (bytes) | 38600386 | 38200000 | 37600376 |
| HTML transferred (bytes) | 19100191 | 19100000 | 19100191 |
| Requests per second [#/sec] (mean) | 3512.04 | 978.95 | 3144.45 |
| Time per request [ms] (mean) | 4.27 | 15.32 | 4.77 |
| Time per request [ms] (mean, across all concurrent requests) | 0.29 | 1.02 | 0.32 |
| Transfer rate [Kbytes/sec] received | 1323.89 | 365.19 | 1154.61 |
Python 3.0.1 serves about 978 requests per second, where Python 2.6.2 does 3512 requests/second.
| Percentage of the requests served within a certain time (ms) | 2.6.2 | 3.0.1 | 3.1a |
|---|---|---|---|
| 50.00% | 3 | 15 | 4 |
| 66.00% | 4 | 15 | 5 |
| 75.00% | 4 | 15 | 5 |
| 80.00% | 4 | 16 | 5 |
| 90.00% | 4 | 16 | 5 |
| 95.00% | 5 | 16 | 5 |
| 98.00% | 5 | 17 | 6 |
| 99.00% | 6 | 19 | 6 |
| 100% (longest request) | 3004 | 3019 | 3006 |
Obviously Python 3.0 does something wrong, talking about factor 4x, it is noticeably slower. But I’m glad to see it is fixed in 3.1 already.
I tried to benchmark using unladen-swallow, the llvm based python project run by google, but was not able to link the code properly. (undefined symbol: _ZTIN4llvm14ModuleProviderE)
As Python 3.1a is almost as fast as 2.6 for our needs, and it can be tricky to port python 2.6 code to 3.x, we will use python 3 as embedded scripting language.