Difference between revisions of "Benchmarking and Load Testing with Siege"

Revision as of 19:54, 18 March 2014

Benchmarking and Load Testing with Siege

Siege is a command line utility which exposes the performance related issues of web based applications. Siege simulates the given number of users who are calling the url; within a given period of time. For example; Siege can tell – "How well a server can perform, if it is called by 200 users within a period of 10 second".

Install Siege on Centos

You will need to install the epel repos to use yum to install siege _{* for instructions to install the epel repos click here}

With the epel repos installed, now you just need to run

yum install siege

Using Siege on Centos

One thing to note, you should never run siege on your localhost against your localhost.

Lets run our first test, 10 connections over 10 seconds

siege -c10 -d10 -r1 -v http://www.briansnelson.com

** SIEGE 2.70
** Preparing 10 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200   0.04 secs:    8438 bytes ==> /Main_Page
HTTP/1.1 200   0.04 secs:    8438 bytes ==> /Main_Page
HTTP/1.1 200   0.04 secs:    8438 bytes ==> /Main_Page
HTTP/1.1 200   0.07 secs:    8438 bytes ==> /Main_Page
HTTP/1.1 200   0.07 secs:    8438 bytes ==> /Main_Page
HTTP/1.1 200   0.04 secs:    8438 bytes ==> /Main_Page
HTTP/1.1 200   0.04 secs:    8438 bytes ==> /Main_Page
HTTP/1.1 200   0.04 secs:    8438 bytes ==> /Main_Page
HTTP/1.1 200   0.04 secs:    8438 bytes ==> /Main_Page
HTTP/1.1 200   0.03 secs:    8438 bytes ==> /Main_Page
done.
Transactions:		          10 hits
Availability:		      100.00 %
Elapsed time:		       10.03 secs
Data transferred:	        0.08 MB
Response time:		        0.04 secs
Transaction rate:	        1.00 trans/sec
Throughput:		        0.01 MB/sec
Concurrency:		        0.04
Successful transactions:          10
Failed transactions:	           0
Longest transaction:	        0.07
Shortest transaction:	        0.03

Lets break down the request we used with siege

• -c10 is the number of concurrent users we want to simulate
• -r1 is the number of repetitions
• -d10 is the delay between each user request
• -v to show the output of each request
• last is the domain/url you wish to siege

After the requests have completed Siege will show you a summary report. This report is documented in the siege man page, which I have reproduced below:

Transactions The number of server hits. In the example, 25 simulated users [ -c25 ] each hit the server 10 times [ -r10 ], a total of 250 transactions. It is possible for the number of transactions to exceed the number of hits that were scheduled. Siege counts every server hit a transaction, which means redirections and authentication challenges count as two hits, not one. With this regard, siege follows the HTTP specification and it mimics browser behavior.

Availability This is the percentage of socket connections successfully handled by the server. It is the result of socket failures (including timeouts) divided by the sum of all connection attempts. This number does not include 400 and 500 level server errors which are recorded in “Failed transactions” described below.

Elapsed time The duration of the entire siege test. This is measured from the time the user invokes siege until the last simulated user completes its transactions. Shown above, the test took 14.67 seconds to complete.

Data transferred The sum of data transferred to every siege simulated user. It includes the header information as well as content. Because it includes header information, the number reported by siege will be larger then the number reported by the server. In internet mode, which hits random URLs in a configuration file, this number is expected to vary from run to run.

Response time The average time it took to respond to each simulated user’s requests.

Transaction rate The average number of transactions the server was able to handle per second, in a nutshell: transactions divided by elapsed time.

Throughput The average number of bytes transferred every second from the server to all the simulated users.

Concurrency The average number of simultaneous connections, a number which rises as server performance decreases.

Successful transactions The number of times the server responded with a return code < 400.

Failed transactions The number of times the server responded with a return code >= 400 plus the sum of all failed socket transactions which includes socket timeouts.

Longest transaction The greatest amount of time that any single transaction took, out of all transactions.

Shortest transaction The smallest amount of time that any single transaction took, out of all transactions

Benchmark with a list of urls

If you want your benchmark test to be more like a real traffic, you can get a list of your urls and have siege use that instead of a single url.

Example syntax

siege -c10 -d2 -r1 -v -i -f briansnelson-urllist.txt

Now I have more than 10 urls in my list, so it will randomly choose the urls.

** SIEGE 2.70
** Preparing 10 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200   0.03 secs:    4925 bytes ==> /Varnish_will_not_restart_WHY%3F
HTTP/1.1 200   0.03 secs:    4937 bytes ==> /Wordpress_Ngnix_Rewrite_Rules
HTTP/1.1 200   0.04 secs:    5163 bytes ==> /Block_Bots_by_User_Agent_String
HTTP/1.1 200   0.03 secs:    4913 bytes ==> /How_to_Import_a_MySQL_Database
HTTP/1.1 200   0.04 secs:    6590 bytes ==> /Remove_Email_Block_Lists
HTTP/1.1 200   0.04 secs:    5152 bytes ==> /Flush_Your_Memcached_Instance
HTTP/1.1 200   0.04 secs:    5698 bytes ==> /How_to_check_Used_IP_on_your_Home_Network
HTTP/1.1 200   0.04 secs:    8505 bytes ==> /Main_Page
HTTP/1.1 200   0.04 secs:    6356 bytes ==> /Crontab_Usage
HTTP/1.1 200   0.03 secs:    4912 bytes ==> /Setup_Varnish_-_Multiple_Ipaddress
done.
Transactions:		          10 hits
Availability:		      100.00 %
Elapsed time:		        2.03 secs
Data transferred:	        0.05 MB
Response time:		        0.04 secs
Transaction rate:	        4.93 trans/sec
Throughput:		        0.03 MB/sec
Concurrency:		        0.18
Successful transactions:          10
Failed transactions:	           0
Longest transaction:	        0.04
Shortest transaction:	        0.03