Time for a Real World Linux/Apache - IIS Comparison

All my real world experience is with MS infrastructure. This trying article is an attempt to take short cuts to broaden my perspective. All of the articles comparing the two leading web environments are outdated, obviously biased, or do comparisons that aren’t what I’m interested in.

My true apples to apples comparison compares strength to strength for the same non-trivial web application requirement. There are so many different (and important) scenarios that I have to make compromises. Here goes…

1) For each run of the experiment use the same 2 64-bit servers, one configured as a file server, one a web server, both on the same subnet. (Expanding the experiment to more and more cores on the web servers might also have interesting results.) I am deliberately not including a database server component.

2) Use Windows 2008 / IIS 7.0 for Windows and the latest and greatest Linux / Apache, each configured as the respective experts deem best.

3) Put a million relatively large (1M) XML files on the file server in a folder structure about four levels deep.

4) Build the Windows app in C#, latest release of .NET (3.5). Try two different configurations for Linux / Apache, one written in Java, the other PHP.

5) Here’s the spec for the app:

a) The request causes the app to read a random XML file from the file server. (Each configuration’s knowledge of the available files must be comparable. Either it is cached at app startup or they have the same means of discovering a readable file path.)

b) Iterate through the entire XML structure assigning the node value to a string variable (for no good reason other than burn CPU cycles).

c) Perform a uniformly defined non-trivial transform on the entire 1M XML file.

d) Respond with a smaller fragment of the transformed XML (I don’t want to test throughput of the response, but the rule for determining the fragment must be uniform).

Measure the throughput, as for any good benchmark, and scale up the request frequency until each configuration “breaks”.

I don’t expect anyone to rush out and perform this experiment. I certainly am not going to, but perhaps something close to this has been done, or someone with real experience may care to speculate on likely outcomes.