Windows vs Linux Security Challenge.

I've always thought Linux to be one of the most secureable operating systems. (Yes, OpenBSD/FreeBSD/NetBSD are right up there too, followed by the not-so-open proprietary Unix-like systems.)

Close to dead last in the list of OSs that are currently in wide use would be Windows. Though there are excellent resources available that will help you secure a Windows installation, the time it takes to do it absolutely right is obscene, and seldom would be done by most administrators, much less average individuals.

To prove my point, I offer the following contest. Pit me against a Windows security guru. Each of us will create a server that is locked down and provides common Internet services. On my box I'll install Linux, and on his he'll install Windows 2000 or WindowsXP, at his discretion.

Ideally, said contest should be held at a security conference, with the monitor and statistics of each participant available for everyone to see as they work.

Necessary Services

• A Web Server, preferably with dynamic-content generating capabilities, such as ASP or mod_perl. No documents need be installed, however all default-install documents/programs must be deleted. In other words, every possible request should return a 404.

• Anonymous FTP Server (read-only)

• Mail Server (able to accept email for itself and send to other Internet machines)

• DNS Server (able to act as a primary for 'OS.example.com' and as a cache for the local network)

• Firewall rules that allow only the above protocols, and any other packets necessary for system administration and normal functionality. (Inbound SSH, DNS Replies, etc.)

The software I'd probably choose would be Apache (mod_perl), DJB's publicfile for anon FTP access, Postfix for the mail server, and DJBDNS for the DNS server/caching server.

The rules

1. Both parties are given identical machines. These machines should have blank hard drives, CD-ROM, floppy, keyboard, monitor, mouse and network card. All hardware must be supported by both a standard Linux kernel and Windows2000 machines.

2. Both machines will be on a LAN switch that has access to the Internet. The machines will share this connection.

3. The contestants will be allowed to have the necessary CD(s) containing their OS. This must be a standard installation CD that anyone can get by walking into a store, ordering online, or downloading over the Internet and burning on their own. It may not have anything custom to this contest, or anything extra or preconfigured.

4. Contestants may not access anything online that they or others have prepared for use in this contest.

5. Any security or installation documentation may be read and consulted as needed.

6. No action may be taken by one machine to interact with the other in any way during or after the installation. In other words, not a packet will hit the wire that isn't honourable. If feasible, the machines can be on separate networks entirely, as long as they have completely identical setup and resources.

7. Play fair. Pretend you're a peon - if they can't do it, you shouldn't either.

Acceptable Network Access

• Anything that is accessed is available to the entire world without need of authentication, registration, IP restrictions, etc.

• Said information/software must be easy to find. Ideally, one should be able to go to http://www.microsoft.com or http://www.debian.org and click their way to the files/information without even entering a search.

• Hidden pages are not allowed.

• Hard-to-find pages are severely frowned upon.

• Pages that were made available specifically for this contest, or seem tailored for this contest are forbidden.

So, what's appropriate downloadable content? A new hotfix that has been released for IIS and is available from the Microsoft security downloads section, for example, is completely fine. A hardening program that is not 100% publicly available and easy to find is not.

What if Microsoft releases such a tool specifically in reaction to this contest? That's great! I'd love for them to make stuff available to make securing Windows easier. However they'd better keep it up to date over time -- otherwise I will consider it unfair play, and not beneficial to their users.

Judging

• Overall time from start to finish.
• Time for initial installation.
• Time spent consulting security-related information, and list of sources.
• Time spent rebooting.
• Time spent 'doing something' vs waiting for downloads to complete.
• Time spent finding and selecting patches/service packs.
• Time the machine is vulnerable on the network.
• Number of patches required.
• Number of reboots required.
• Amount of install that must be completed at console, and could not be done remotely.
• Software packages required, and total cost of the server setup (software + configuration - hardware is irrelevant)
• Brief security history (latest vulnerabilities, and impact) of the network-accessible software (web/dns/ftp/etc) installed.

My hope is to provide a side-by-side comparison to let us see the installation and securing process of Linux and Windows machines. We should be able to infer some best-practices for both worlds, and see where current systems are deficient.

This is meant to be beneficial to both Linux and Windows administrators. This is not about 'bragging rights'. Yes, I do expect Linux will prove itself easier and quicker to secure, but I hope to have some useful data that users and administrators can use to secure their own machines and make informed choices.