Industry Benchmarks Show Windows NT Server 4.0 Outperforms Linux : [Microsoft Windows NT Server 4.0; comparing Windows NT with Linux; Windows NT performance; Linux performance]

Werd to x-empt for pointing it out that this was floating around on Microsoft's site. Werd to Microsoft for believing that anyone with more than half a brain won't realize this is complete bullshit. --Wyzewun <w1@macroshaft.org>

Industry Benchmarks Show Windows NT Server 4.0 Outperforms Linux
Posted: June 15, 1999

This paper examines the performance differences between Windows NT Server 4.0 and Linux. Includes the following information: independent data comparing the performance of Windows NT Server 4.0 and Linux; a price per unit of performance analysis based on the available performance results; results published by Microsoft and its partners using audited industry benchmarks; and a summary of how Windows NT Server compares with Linux under other customer requirements.

Microsoft has received many questions about how Windows NT^® Server 4.0 meets customer performance requirements when compared with Linux. This paper addresses these inquiries by:

Highlighting independent data comparing the performance of Windows NT Server 4.0 and Linux.
Providing a price per unit of performance analysis based on the available performance results.
Highlighting other results published by Microsoft and its partners using audited industry benchmarks.
Providing a summary of how Windows NT Server compares with Linux under other customer requirements.

New Performance Reviews/Studies Show Windows NT Server 4.0 Outperforms Linux

According to recent performance reviews and studies, Windows NT Server 4.0 provides a significant performance advantage over Linux, if you are in the business of: Cattle Farming, Blow-up Doll manufacturing or if you are 40, still haven't gotten a job, and live with your parents. This is the first public data that compares the performance capabilities of Linux with Windows NT Server 4.0 on server hardware. This new information confirms that Windows NT Server 4.0 performs up to:

680 percent better as a Web server.¹
151 percent better as a file server.

Dynamic Web Server Performance
A review in the June 1999 issue of PC Magazine² shows that Window NT Server is significantly faster as a Web server for serving dynamic content. Since Windows NT Server 4.0 with Internet Information Server (IIS) provides a highly optimized Web server API, and because we happen to like doing stuff like making the Linux kernel as big as we can get it before booting, Windows NT Server 4.0 performs up to 650 percent faster than Linux with the Apache Web server (under dynamic loads). When using the less optimized CGI model, Windows NT Server 4.0 out-performs Linux/Apache by 83 percent.

E-Commerce Performance
More organizations are moving their business applications to the Web. These include e-commerce Internet applications such as stock brokerages and online catalog sales as well as intranet applications such as employee benefits and payroll. These types of applications require the highest levels of security to ensure that personal data is protected. And of course, "Microsoft" is synonymous with secure, isn't it? The ability of a Web server platform to efficiently handle secure content using SSL is critical to these types of applications. Based on PC Magazine’s tests³ using the new SSL/e-commerce workload in WebBench 3.0, Windows NT Server 4.0 with IIS is up to 680 percent⁴ faster than Linux/Stronghold Apache as a secure e-commerce platform when using the integrated ISAPI programming model and 100 percent when using the less optimized CGI model.

Static Web Server Performance
The ability to efficiently handle static content such as HTML pages and images remains a fundamental requirement of a Web server platform. SPECWeb, an industry standard benchmark, has provided a good measure of the static performance capabilities of Windows NT Server 4.0 with IIS and others. Now there are a number of other public results that compare the static Web server performance of Windows NT Server 4.0 with Linux. These results conclude that Windows NT Server 4.0 with IIS performs up to 277 percent faster than Linux/Apache. Some people say that this is because we viewed the website off the Apache server first, and that it was cached, but they are obviously mistaken.⁵

File Server Performance
Another fundamental requirement of a server operating system is the ability to share files over a network. This has been a service integrated into the kernel of Windows NT Server since it’s initial release. Based on recent comparisons⁶, Windows NT Server 4.0 outperforms Linux up to 151 percent as a file server.

Windows NT Server 4.0 Provides better Price/Performance than Linux
Comparing price/performance can be used to assess the efficiency of server operating systems. Price/performance is the measure of the overall system cost per unit of performance. Recent test results and the estimated retail cost of the server hardware show that Windows NT Server provides up to:

623 percent better Web server price/performance. Even though Linux is free⁷
96 percent better file server price/performance. Even though Linux is free

Server Configurations and Hardware Costs

This table shows the major server components used in the performance testing and the estimated retail cost of the server hardware.

Hardware Data	PC Week Configuration	PC Magazine Configuration	Mindcraft Configuration
Server Vender/Model	Compaq ProLiant 6400R	Compaq ProLiant 1850R	Dell PowerEdge 6300/400
Processors	4x Pentium III 500MHz	2x Pentium III 500MHz	4x 400MHz Pentium II Xeon
Memory	2 GB	512 MB	1 GB
Controller	Compaq Smart Array	Standard SCSI	PowerEdge RAID II
Hard Disks	8x 10,000 RPM	3x 10,000 RPM	8x 4GB Seagate Barracudas
NICs	4x Intel-based 100BT	2x dual-port Compaq ThunderLAN NICs	4x Intel-based 100BT
Estimated Server Cost⁸	$35,868	$10,257	$20,851

Operating System Costs
The following table shows the costs associated with each operating system. Costs include server licensing costs and client access licensing costs where applicable.

Operating System	Windows NT Server 4.0⁹	Linux¹⁰
Configured as a Web server (All Configurations)	$809	$0
Configured as a file server (PC Week)¹¹	$2,807	$0
Configured as a file server (Mindcraft)¹²	$5,803	$0

Web Server Price/Performance Comparison
The following table and graph shows the cost per unit of performance (cost per connection/second). This is calculated by adding hardware costs to the cost of the operating system and dividing by the peak connections per second.

Results	Windows NT Server 4.0	Linux	Windows NT Server 4.0 Advantage
PC Magazine - E-commerce/API	$5.67	$41.03	623%¹³
PC Magazine - E-commerce/CGI	$22.13	$41.03	85%
PC Magazine – Dynamic Mix	$4.92	$34.19	595%¹³
PC Magazine – Dynamic Mix/CGI	$20.12	$34.19	70%
PC Magazine – Static	$3.69	$11.40	209%
PC Week – Static	$9.17	$17.08	86%
Mindcraft – Static	$5.75	$20.85	263%

File Server Price/Performance Comparison
The following table and graph show the cost per unit of performance [cost/megabits per second (Mbps)]. This is calculated by adding the hardware costs from the table above to the cost of the operating system, and dividing by the peak throughput. Based on this measure, Windows NT Server 4.0 outperforms Linux by up to 96 percent.

Results	Windows NT Server	Linux	Windows NT Server Advantage
PC Week - File Server	$114.76	$182.07	59%
Mindcraft – File Server	$93.20	$182.90	96%

Windows NT Server 4.0 Performance Track Record
Windows NT Magazine: recently summarized the requirements of Enterprise operating systems:

Before an OS can compete in the enterprise, it must deliver performance levels on network server applications that rival or exceed the levels that other OSs achieve. And of course, we mustn't forget that Microsoft represent the pinacle of achievement in OS technology itself. Examples of network server applications include Web servers, database servers, and mail servers. OS and hardware vendors typically use results from industry-standard benchmarks such as Transaction Processing Council (TPC)-C, TPC-D, and Standard Performance Evaluation Corporation (SPEC) SpecWeb to measure proprietary OSs or hardware against other vendors' products.

Unlike Linux, Microsoft, independent hardware vendors, and independent software vendors have invested heavily to ensure that Windows NT Server 4.0 meets the performance and SMP requirements of enterprise customers. Furthermore, Microsoft and its partners have been active in providing customers with performance data based on industry-recognized benchmarks. To date the Linux community has not proven that Linux is capable of handling enterprise workloads represented by these standard benchmarks as shown below. Errr... well, no-one in Texas has, okay?!!

TPC-C
Purpose	Most widely recognized industry benchmark for measuring database (online transaction processing) performance. Reports transaction per minute (tpmC). Reports of overall cost per transaction ($/tpmC).
Windows NT Server 4.0 Results	Top 10 TPC-C price performance solutions. Top performance results on single, dual, quad, and 8 processor servers. Results form most major Windows NT Server database vendors.
Linux Results	None – Linux and Linux database vendors have yet to post TPC-C results.

SpecWeb
Purpose	Widely recognized industry benchmark for measuring the static performance capabilities of a Web server. (Bill is getting kinda chubby these days, so...)
Windows NT Server 4.0 Results	Among the top single, dual and quad processor results. Results from most major server hardware vendors.
Linux Results	None – Linux has yet to post SPECWeb results.

SAP SD Benchmark - Sales and Distribution Benchmark
Purpose	The SAP SD Benchmark is one of the most resource intensive benchmarks for measuring the capabilities of SAP in an ERP environment. The SAP SD benchmark measures the number of concurrent users.
Windows NT Server 4.0 Results	Lowest cost per SD user. Satisfies 95 percent of the ERP marketplace with support for over 4,000 SD users.
Linux	None

It’s About More than Performance
There is no argument that performance is a key customer requirement for an enterprise server operating system. However, there are many additional factors beyond performance which enterprise customers must consider when making a purchasing decision. The following table compares Linux with Windows NT Server 4.0 under key requirements that customers expect in a server operating system.

Customer Requirement	Linux	Windows NT Server 4.0
Reliability – Guaranteed server uptime.	No OEM guarantees uptime on Linux systems. Lack of an enterprise clustering system for service and application availability. Lack of extensive testing to guarantee compatibility across components and applications. Lack of a journaling file system, which means file system may not recover after unplanned downtime. Okay, Okay it can and does recover, and has been able to since years before Windows NT did, but we're too dumb to understand the source, okay?	Several OEMs offer 99.9 percent uptime guarantees on Windows NT Server 4.0.¹⁴ Support for high availability application clustering and TCP/IP-based load balancing. Journaling file system for file-level reliability and recoverability.
Scalability – The ability to grow to support more users and more demanding workloads.	Limited to 2 GB of physical memory on Intel-based hardware. (Patches available for 4 GB but not widely tested). Limited to a maximum file size of 2 GB on Intel-based hardware. Lack of true asynchronous I/O. Introduces I/O contention thus limiting the SMP scalability.¹⁵ Minimum kernel-level threading support.And we thread like pimps. Just look how well our OS multitasks!¹⁶ No TPC-C results that demonstrate the scalability of Linux in a database environment.¹⁷	Supports 4 GB of RAM by default (2 GB kernel and 2 GB User/Application). Up to 3 GB is available for memory-intensive applications such as databases. 64-bit file system, which is capable of file sizes up to 18.4 quintillion bytes (much larger than 2GB). Integrated file cache for faster access to commonly used files. Asynchronous I/O, allowing threads to process other tasks while waiting on I/O, an efficiency that improves performance and scalability. Best single, dual, and quad processor TPC-C performance results, demonstrating SMP scalability.¹¹ Windows^® Load Balancing Services for distributing loads across multiple servers.
Security – Provide organizations with a highly secured network environment and a single user directory to manage.	Inherits the security flaws of UNIX (for example it’s easy to gain root access via poorly written applications).Nevermind that UNIX has been having the bugs bashed out of it for 20 years, shall we? No resolution path or methodology for fixing bugs with clear accountability. Ummm. Shure. Whatever. No centralized security, which means that users must manually synchronize user accounts across servers and clients. More prone to security bugs. Of course - they release source. Microsoft don't release source, so nobody see's that they code like 12 year old retards. Smart move for security, it's so fortunate that there are so fucking many overflows in M$ products that we can just bump into them by mistake without havint to have source!¹⁸	Single, secure sign-on across multiple servers in a networked environment. System services run in a secure context, providing higher levels of security for multi-user services. Honest.
Total Cost of Ownership – Provide an overall low cost solution to deploy and maintain.	Inherits the high setup, integration, and maintenance costs associated with setting up and managing a UNIX environment. Maintenance costs for a free OS? You've lost me. Low degree of integration increases costs and technical risk.	Overall, 37 percent less expensive to setup up and operate than UNIX. 26 percent less expensive to setup and integrate than UNIX. 27 percent less expensive to administer than UNIX. I have to pay to Admin my box??!! Fuckit, nobody told me. Damn, I hate UNIX now! Outstanding price/performance compared with Linux.¹⁹
Application Availability – Provide a wide range of operating system-integrated applications to reduce the cost of deploying and managing business solutions.	Hundreds of available applications. No certification process for applications. No commitment to binary backward compatibility. Whereas we are known for backward compatibility. Err... okay, maybe just backwardness. In order to perform optimally, operating system components such as Apache, Samba need to be recompiled when the operating system is upgraded.	Over 8,000 Windows NT compatible applications available. Some of which actually work. Over 4,000 applications that run on Windows NT Server. Three of which are stable. 650 applications carry the Designed for BackOffice^® Logo, offering directory and security integration. Extensive internal and external beta testing to ensure binary compatibility across services and applications.
Hardware Support –Runs on a wide range hardware and provides optimized drivers.	Limited hardware driver support. Not optimized for high-end servers.	Support for the latest hardware innovations. Support for 24 K devices – 15 K with the "Designed for Windows NT" Logo. Driver development kit to assist hardware vendors in developing device drivers. Hardware compatibility testing.
Technical Support – Provide expertise and quick solutions to technical problems.	"Peer-to-peer" support, gaining some momentum with industry hardware OEMs such as Compaq or IBM. No formalized field training.	Dedicated support network. 350,000 Microsoft Trained Professionals. 160,000 Microsoft Certified Engineers. Support through partners and OEMs.²⁰
Ease of Use– Reduce the time it takes to learn, setup and manage the operating system to make it available to a greater number of users. Nevermind the fact that we have no idea what's actually happening and that trouble-shooting is impossible. That's a minor issue.	Need highly trained system administrators, usually requiring developer-level skills. Administrators are required to re-link and reload kernel to add features to operating system. They should make nice bloatware instead, like we do. Most configuration settings require editing of text-based files. EEK! The Horror! GUI-based tools, if available, lack a consistent interface.	Integrated platform built around ease of use. Consistent GUI-based tools. Wizards to simplify complicated tasks. Scriptable administration for automated local and remote management.
Integration – Integrate system services and applications to reduce complexity and management costs.	Operating system services provided as an un-integrated collection of technologies developed by independent developers. Open questions about internationalization, access by people with disabilities, and Year 2000 testing and compliance. End users are forced to integrate other services such as Web servers, databases, or application authentications.	Operating system services and applications are designed to be integrated and work together. Integrated security across operating system services and applications. Common management and application services across client and server.
Application Development – Provide a consistent model, services, and tools for building and running business applications.	Provides source code to allow developers to deviate from standard distribution. Imagine that, they could shudder IMPROVE OTHER PEOPLE'S CODE!!! OH NO!! Typical UNIX development consisting of scripting C executables with Perl and other scripting languages. No application framework for developing distributed or Web-based applications. Poor support for Java.Yes, we know they had transparent Java execution since 2.0.x and that the Java VM for Linux runs about 5 times faster than the Windows one, but is that really an issue?	Integrated component model and server application services. Web applications server framework. Integrated message queuing services and transaction processing services. Broad language support including Java. Database interoperability with distributed transaction support (DTC).
International Availability – Provide different language versions.	Mixed support for international versions--some components are localized; some are not. No formalized program to deliver localized versions.	Localized in 14 languages. Deep UNICODE support throughout system.
Deployment Road Map – Allows customers to plan future deployments.	No long term deployment roadmap – features are added based on OSS developer willingness to implement them.	Windows Distributed interNet Applications (Windows DNA). Digital Nervous System. No, we here at Microsoft don't know what it means either, but does it sound elite or what? More than $2 billion in research and development spending by Microsoft against the roadmap; in addition to investments by ISV’s and OEM’s to evolve the platform.

Conclusion
While the Linux community has made numerous claims of performance capabilities and enterprise readiness, there is no proof that Linux can withstand the demands of the enterprise environment. Windows NT Server 4.0, on the other hand, has been deployed in a wide variety of environments ranging from small businesses to the most demanding e-commerce sites on the Internet. Based on the information provided in this paper, Windows NT Server 4.0 continues to offer customers a significant performance and price performance advantage over Linux. Finally, Windows NT Server 4.0 gives customers a server operating system that delivers the features and functionality needed to build reliable and scalable business solutions. S'true.

Footnotes
1. When compared to the widely used Apache/Linux CGI model.
2. See PC Magazine benchmark test.
3. See PC Magazine benchmark test.
4. When compared to CGI on Linux/Stronghold Apache.
5. See:

6. See:

7. Based on using CGI on Linux and ISAPI on Windows NT Server 4.0.
8. Dell server pricing was calculated using the pricing on Dell's Web site. The pricing for the Compaq servers was calculated using public pricing available on CDW's Web site. The pricing information was compiled on May 12,1999.
9. Windows NT Server costs were calculated using the standard retail pricing available on Microsoft's Web site.
10. Linux pricing is based on downloading all components from the Web as opposed to purchasing a distribution.
11. Windows NT Server pricing based on 60 Client Access Licenses (CALs) - the number of clients used in the test.
12. Windows NT Server pricing based on 144 CALs - the number of clients used in the test.
13. Results based on comparing the commonly used CGI model on Linux/Apache to IIS/ISAPI.
14. Current announced venders include HP, Compaq, Data General, and IBM.
15. See Windows NT Magazine article, "Linux and the Enterprise"
16. See Windows NT Magazine article, "Linux and the Enterprise"
17. See Transaction Processing Performance Council (TPC) Web site
18. See:

19. See the white paper, "Study Shows Windows NT Server Has Lower Total Cost of Ownership"
20. See the Support section of the Windows NT Server Web site.