You’ve got to give Microsoft credit for their success over the years. They’ve maintained dominance over the mainstream market for a long time, and Bill Gates is a multi-billionaire.
Like all major players, when you’ve established dominance in a given market, you seek to maintain it. This is possible by providing newer and “better” products. Because of competition, you’re always in a rush to get your product out there as quickly as possible without losing any perceived quality.
Windows XP was built taking what they learned from Windows 2000, just as Windows 2000 was built taking what they learned from previous versions of the Windows NT line. Windows NT was a break from the previous incarnations of Windows (which were built on top of 16-bit DOS). It took the then new Win32 API and threw in a compatibility mode for Win16.
Windows 95 was created to use the Win32 API while maintaining maximum compatibility with all 16-bit code from previous versions of Windows/DOS. Maintaining compatibility meant keeping some undesirable elements from those previous versions.
Windows NT 4.0 was produced, and used the Windows 95 GUI on top of the NT kernel. Windows 98 was a “minor” revision to Windows 95 which increased stability and performance. Windows 2000 was major upgrade in the NT line (originally known as NT 5.0), and Windows Me was the short-lived (rightfully so) version of Windows before Windows XP would take the performance and reliability aspects of the NT line and combine them with the user-friendliness of the DOS line.
Finally, the two lines merged (or rather, the NT line took over control) when Windows XP came out. The trouble of maintaining compatibility in both lines worked out. The DOS line was abandoned for it’s relative unreliability, and the NT line got a face lift.
The lesson learned is that maintaining compatibility is a necessary evil. Even if what we have now is junk in comparison to the new stuff, we can’t just throw it away. Microsoft maintained dominance by developing two lines of their operating system, one with minimal compatibility with older software, one with greater compatibility. The line that was more compatible, but less reliable, was eventually retired, and the less compatible line took over.
For as much as you love to hate it, Microsoft did the right thing. However, now that we’re stuck with this NT line, we’re stuck with its own little annoyances and big headaches. Perfection might not be around the corner, but as long as progress is in the making, it’s not entirely bad.