Apache, Nginx, Lighttpd, Monkey, and BarracudaDrive Speed Test

Benchmarking web servers can be difficult since few computers provide the exact same CPU speed. It is for this reason the Raspberry Pi is great for benchmarking since non modified Raspberry Pis will provide the same speed. A standard (non overclocked) Raspberry Pi with its ARM processor runs at 700 MHz.

Note: The Mako Server and BarracudaDrive are both derived from the Barracuda Application Server library and the benchmarking results are identical for these two products.

For the tests we used Siege as it is a great web server benchmarking tool for this type of testing. Siege was run on a Linux powered laptop and connected to the Raspberry Pi via a standard home router.

All tests show how many transactions took place during our 2 minute test. We only tested with 200 concurrent connections; i.e. we instructed the Siege tool to open 200 concurrent connections when running the server siege.

Test 1 - Fetching a small static 177 bytes html page:

ServerTransactions
BarracudaDrive53882
Apache 26624
Lighttpd31807
Monkey39310
Nginx39449

Test 2 - Fetching a large static 179Kb JPG file:

ServerTransactions
BarracudaDrive4881
Apache1390
Lighttpd1357
Monkey1338
Nginx1396

We performed the above BarracudaDrive tests. The results for the other servers above are from Jeremy Morgan's RPI benchmarking.

Jeremy Morgan only tested with static content, but we ran two additional tests where the server responds with dynamically generated content. The following shows the output from running the above two tests and the output from fetching dynamically created content.

Running test 1 - Fetching a small static 177 bytes html page:

$ siege -b -t2m -c200 http://192.168.1.175/file.html

Lifting the server siege...      done.

Transactions:                  53882 hits
Availability:                 100.00 %
Elapsed time:                 119.42 secs
Data transferred:               9.10 MB
Response time:                  0.44 secs
Transaction rate:             451.20 trans/sec
Throughput:                     0.08 MB/sec
Concurrency:                  199.55
Successful transactions:       53882
Failed transactions:               0
Longest transaction:            9.46
Shortest transaction:           0.03

Running test 2 - Fetching a large static 179Kb JPG file:

$ siege -b -t2m -c200 http://192.168.1.175/large.jpg
[error] socket: read error Connection reset by peer sock.c:473: Connection reset by peer

Lifting the server siege...      done.

Transactions:                   4881 hits
Availability:                  99.59 %
Elapsed time:                 119.19 secs
Data transferred:             830.77 MB
Response time:                  3.59 secs
Transaction rate:              40.95 trans/sec
Throughput:                     6.97 MB/sec
Concurrency:                  147.00
Successful transactions:        4881
Failed transactions:              20
Longest transaction:          119.02
Shortest transaction:           0.05

We had a number of errors for the above, which indicates that the RPI is having a hard time serving all the requests. It could also be that our router is having a hard time. We used a basic home router for the test. Jeremy also had errors, but we do not think these errors are related to the servers being used. It is more likely related to the RPI hardware or the router.

The above tests require that the /etc/init.d/bdd script is modified since the default script limits BarracudaDrive to 80 connections. We changed "BDCONNECTIONS=80" to "BDCONNECTIONS=1024"

We also stopped the included CMS and created a new root application for the above tests.

We are a bit surprised by the results since BarracudaDrive is an application server and not a web server. Typically, application servers should be slower than web servers since they have more housekeeping to do.

Since BarracudaDrive is an application server, it can also run server side scripts similar to PHP.

We used the following Lua Server Pages (LSP) script for the benchmarking.

<?lsp response:write"Hello World" ?>

Although the above is a very basic LSP script, it still triggers a fairly extensive set of actions on the server such as compiling the LSP script to Lua source code, compiling Lua source code to Lua byte code, and then finally executing the byte code using the Lua virtual machine. There is no caching so the above sequence is performed for each request.

Running test 3 - Fetching dynamically created content:

(The following shows the benchmark results for running the above LSP script)

$ siege -b -t2m -c200 http://192.168.1.175/test.lsp

Lifting the server siege...      done.

Transactions:                  45577 hits
Availability:                 100.00 %
Elapsed time:                 119.15 secs
Data transferred:               0.48 MB
Response time:                  0.51 secs
Transaction rate:             382.52 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  195.35
Successful transactions:       45577
Failed transactions:               0
Longest transaction:           45.15
Shortest transaction:           0.03

From the above test, we can see that BarracudaDrive can serve dynamically created responses faster than the other servers are able to send back the small static html file in "test 1" above.

We looked for a PHP speed test and could not find one. BarracudaDrive's Lua Server Pages (LSP) is similar to how PHP creates dynamic content. Maybe you have done a PHP speed test on Raspberry Pi? Can you manage 45577 transactions in 2 minutes (382 transactions a second) on a Raspberry Pi using PHP?

Running test 4 - Content Management System Speed Test:

We ran one final test, where we disabled the LSP root application we used for the tests above and then re-enabled the BarracudaDrive Content Management System (CMS). The following test shows how many transaction the CMS is capable of handling in two minutes.

$ siege -b -t2m -c200 http://192.168.1.175/

Lifting the server siege...      done.

Transactions:		       10106 hits
Availability:		      100.00 %
Elapsed time:		      119.35 secs
Data transferred:	       53.69 MB
Response time:		        2.34 secs
Transaction rate:	       84.68 trans/sec
Throughput:		        0.45 MB/sec
Concurrency:		      197.90
Successful transactions:       10106
Failed transactions:	           0
Longest transaction:	        5.99
Shortest transaction:	        0.06

The CMS is not a small Lua Server Pages application. There's a whole lot of stuff going on for each HTTP transaction including opening and searching for the page content in the CMS SQL database. Still, we manage to serve 84 pages a second on a Raspberry Pi.

About BarracudaDrive:

BarracudaDrive is a Barracuda Application Server derivative product. BarracudaDrive is built by using the standard Barracuda Application Server source code library.

Posted in News Thursday, March 28th, 2013