Recently, a small testing company published the results of a privately commissioned web server performance test in which Micro$oft’s NT and IIS beat the pants off of Apache running on Red Hat Linux 5.2. It subsequently came to light that the test was commissioned by Micro$oft. And perhaps needless to say, it has touched off quite a furor.

In fact, so much furor-touching has it done that Mindcraft has taken the unusal step of proposing a second benchmark, this time inviting in participants from both the Micro$oft and Open-Source camps. The idea is that if experts from both sides are brought in to do their best at tuning their respective systems for optimum performance, then the cacophony of voices crying “foul” will quiet somewhat. (One has to presume that underlying this is the fervent hope that NT will win anyway, and that this will provide some annoyed Redmondian parties a brief respite from the past few months of glowing press coverage about the emergence of Linux.)

But what should really expect from this re-test? Matthew Chappee makes a very salient point in passing in his article (not currently available), “What if Microsoft is right?“, when he says:

“This is a golden opportunity for the Open Source movement. … There will be areas where Linux will rule, but there will also be areas where Linux will lag. I can guarantee this, within one month of seeing these benchmark results, Linux will ‘correct’ and smash NT on the next pass. ”

I can’t predict the outcome of this second test. But the immediate results are only of passing interest to me. The real power of the Open Source movement is not revealed by a snapshot of results, frozen in time at a given moment. The Open Source model is superior to other models, but NOT because all Open Source software is always better at any given time than an older, more mature, better-funded closed-source competitor (it’s not). It’s superior because continuous improvement is built in. Open-Source software behaves like an organism that desires to survive, and is almost infinitely adaptable. Challenges make it stronger, because it forces the organism to morph and evolve and adapt and improve to survive.

And that is the real reason that the Open Source movement is a threat to any closed-source competitor. It evolves so much more rapidly than any observed closed-source model in existence, that it’s simply impossible to stay ahead of it indefinitely.

So from where I sit, this “open benchmark” will certainly be of interest. But as an Open Source advocate, what I’m looking forward to are the changes and releases it engenders within the next 3-6 months, and the tremendous performance gains we’re likely to see in that time period. Within a matter of days after the initial results were announced, I have seen the Linux community adapting to the charge that tuning information is hard to find by beginning to collect and publicize extensive collections of performance tuning tips and resources. Code is already being scrutinized, and changes are already underway to meet this latest challenge.

The job of those who have an interest in the advancement of all that Open Source represents will be to point this out 6 months from now, and to show the tremendous evolutionary gains over that period of time. It will be our job to point out that this is the standard course for active projects in the Open Source community.

This is easy to overlook, and is going to be missed or ignored by many in the mainstream press. If we’re not careful, it will be easy for the Open-Source community to let this slip between the cracks and focus on some new challenge or FUD-attack or whatever is current around the end of the year. But we need to bring this back up after a few months and to point out that, as it always does, Open-Source software continues to raise the bar.

So let’s do the test, let’s talk about the results, let’s crow loudly or eat crow as the immediate result may require. But let’s
keep working, keep focusing on the real strength of the movement. And let’s keep writing software that quietly overcomes and displaces all contenders. Meanwhile, Micro$oft will continue to slip on their promised delivery of Windows2000, will continue to throw money at a development model that is ultimately doomed. They will continue to lose ground in the Internet server market, and increasingly on the desktop by the end of the year, and they’ll continue to wonder why. “But we were faster in those test results we bought back in April,” they’ll say. “We won, didn’t we?”