In-Depth

SQL 2005: The Integrated Stack Is Back

Is all this integration among Microsoft components good for customers?

Think of SQL Server 2005 as a bulldozer ready to reshape the industry. Now pull back your mental camera and imagine it as the leader of a phalanx of Microsoft bulldozers grinding across the IT landscape, heading for more hills to flatten.

For a product getting its first major revision in five years, SQL Server carries tremendous momentum. Surprisingly, much of that momentum doesn't flow from the technical fine points of the database server. Instead, SQL Server 2005 is driven by deep hooks into other Microsoft server and desktop products. Microsoft decided to break with the industry practice of selling separate data warehousing and business intelligence (BI) components for high premiums. Instead, the company has bundled these features into SQL Server itself, yielding a radically lower price structure that forces database managers to take notice.

The 2005 version is not the first release of SQL Server to employ hooks and bundling as a market strategy, but it could be among the most effective. Clearly, the integration has benefits for Microsoft; but is it good for you?

During the Nov. 7 launch of SQL Server 2005, Microsoft placed heavy emphasis on the integration among Microsoft products and between components of SQL Server itself.

"Through deep collaboration with our customers and partners, today we're delivering powerful new platform capabilities with unprecedented integration between the server infrastructure and development tools," Microsoft CEO Steve Ballmer said.

Cross Product Integration
The triple product launch of SQL Server 2005, Visual Studio 2005 and BizTalk Server 2006 on the same day highlights the deep integration that lurks within Microsoft's product set. Over the last decade, Microsoft has made hundreds of choices resulting in the increasingly tight integration of SQL Server with Windows Server, Visual Studio, BizTalk Server and Office.

Nowhere is the push to integrate more evident than with Windows Server. There was a time when such an approach wasn't a given. In June 2000, Judge Thomas Penfield Jackson ordered that Microsoft be split into an applications company and an operating system company, forcing questions about porting SQL Server to other platforms. When an appeals court overturned Jackson's judgment, however, it put such thoughts firmly to rest. Today, the platform lock-in between SQL Server and Windows is assumed.

Other forms of integration abound. The completely new version of Visual Studio being released with SQL Server 2005 is so tightly integrated that the products' features are hard to separate. The Common Language Runtime, arguably the most important new feature of the "Yukon" release, is so deeply coupled with Visual Studio development tools and languages that it's difficult to talk about the database without the Integrated Development Environment. The tight relationship between SQL and Visual Studio produces much of SQL Server's key competitive advantage as a rapid and relatively painless database development platform. This trend is hardly new, but it has been carried forward aggressively with the 2005 release of SQL Server.

With the Microsoft Dynamic System Initiative taking effect across the Windows Server System, SQL Server 2005 had to clear a number of Microsoft integration hurdles to qualify for a release to manufacturing. These include enabling management via a product-specific Microsoft Operations Manager 2005 Management Pack, the ability to run on Microsoft Virtual Server, support for the Windows Volume Shadow Copy Service for backup and recovery, and support for Microsoft's revamped patching infrastructure.

To Market, to Market
Five years after SQL Server 2000 put Microsoft on the database map, the company faces a whole new set of challenges.
By Michael Desmond

All that is old is new again. It was ten years ago that Microsoft truly entered the database system market, adapting program code it had purchased from leading database vendor Sybase for Microsoft SQL Server 6.0 for Windows NT. IBM, Informix, Oracle, and Sybase dominated the market. A decade later Informix is gone, Sybase is a shadow of its former self, and Microsoft -- with the launch of SQL Server 2005 -- is poised to challenge for DBMS leadership.

"It was kind of an interesting journey," recalls Ilya Bukshteyn, SQL Server director of communications. "When we entered this space we were actually tenth in a field of nine."

How times change. Bukshteyn's field of nine has narrowed to three -- IBM, Oracle and Microsoft. SQL Server leads all products in both unit sales and year-over-year revenue growth, though it trails in total revenue due to a sharply lower price tag. Now with SQL Server 2005, the company has set its sights higher, challenging Oracle and IBM with a product that can scale to multiple terabytes, while aggressively integrating with the Microsoft stack.

Timing Is Everything
The database formerly known as Yukon may be almost two years overdue, but Yankee Group research fellow Laura DiDio says the software could be arriving just in time. She points to an August survey that indicates that 48 percent of businesses are on a four to six year replacement cycle for Windows database software.

"We're on the precipice in the next 12 to 15 months of a very, very large mass migration. A lot of people last did a major overhaul of their networks back in 1999, back before year 2000," says DiDio, who describes SQL Server as "well positioned."

Perhaps most surprising is that SQL Server 2005 arrives with a serious head of steam. According to IDC, SQL Server 2000 was the fastest growing database system in 2004, with sales rising 22 percent to just over $2 billion.

That performance has helped firmly entrench Microsoft near the top of the database market, says Carl Olofson, research director for Information Management and Data Integration Software at research firm IDC. His research shows Microsoft capturing 13.4 percent of the database revenue pie. That's a far cry from 2000, when Microsoft clung to 6.7 percent of the market on $840 million in sales.

The timing may also be right for Microsoft to make a serious run at Oracle, says DiDio. "Oracle might be a little softer and more vulnerable to competition, because of the very bloody and protracted battle to buy PeopleSoft and then Siebel."

"I think we're fully engaged," responds Mark Townsend, senior vice president of database product marketing at Oracle. "As a database guy, I would welcome a little distraction."

Alan Zeichick, editorial director of SD Times, contends that even a distracted Oracle is a capable opponent: "I think [it's] massively distracted, but I don't think [it's] ceding anything. [It] continues to pound away, [it] keeps adding new features and adding to the API within Oracle."

Chutes and Ladders
The SQL Server 2005 launch has made one thing abundantly clear: database competition is no longer about high-end and low-end. SQL Server 2005 covers the entire waterfront, from multi-terabyte data warehousing applications to hobbyist projects based on freely available tools.

The SQL Server 2005 family includes four variants: Enterprise Edition, Standard Edition, Workgroup Edition and Express Edition. New to the family are the Workgroup and Express Edition flavors, tailored for departmental and other less demanding environments. The free Express Edition is designed to help fend off increasingly capable open source database offerings, including MySQL 5.0, which was released in October. Oracle recently announced a similar no-cost offering, Oracle 10g Express Edition.

But it's at the high end where Microsoft must prove its mettle. Richard Winter, president of database research firm Winter Corp., says Microsoft is making strides, as evidenced by SQL Server's increasing presence on Winter Corp.'s TopTen database program. But he says the company has a ways to go. "Microsoft is now present in the survey in the range of one to 20TB approximately, but the biggest database is ten times that size," Winter says.

Zeichick argues that the tightly integrated SQL Server stack -- the focus of much of attention in this release -- could work against Microsoft at the data center level. "BizTalk, Visual Studio, Exchange, SharePoint -- you put it all together and it's one big log -- a black box if you will. Big enterprises don't think that way. They really want that n-tiered approach."

Microsoft has plenty of experience scaling technology to the enterprise, but Winter warns that the climb up the database ladder may be more difficult than what the company faced with Windows. Microsoft may have found that out when it was forced to forego clustering functionality in the initial release of SQL Server 2005. The delay leaves IBM DB2 and Oracle 10g with some breathing room when selling into highly scaled environments. Microsoft expects to add clustering in a future release.

Squeeze Play
Even as SQL Server 2005 guns for new heights, Microsoft must defend against burgeoning threats at the low end. Open source databases like MySQL and PostgreSQL have grown increasingly capable and sophisticated, while growing adoption of Linux has compounded the challenge.

"They are progressing and they have a kind of disruptive effect on the market," says Olofson of open source database products. "They can kind of work up the stack with a different revenue and cost structure, if you will, than the large vendors have."

MySQL 5.0, for instance, went to production in October and now offers critical features like triggers and stored functions, as well as expanded compatibility and improved security. Microsoft's response has been SQL Server 2005 Express Edition, a limited version of the database available for free download. Express Edition offers many of the integration hooks that make the full product so attractive to mid-range deployments, but caps database size at 4GB. Oracle and IBM each offer Express versions of their database products as well.

As SQL Server 2005 matures, Olofson says IBM and Oracle can no longer count on customers migrating to their databases from SQL Server. That makes preserving what Olofson calls the "on ramp" of young developers all the more critical. "They want to give them some alternative to put them on an Oracle track instead of on a Microsoft track," says Olofson.

Now Microsoft, facing low-end competition from open source alternatives, finds itself doing the same thing.

Of course, SQL Server is married to one inescapable part of the Microsoft stack -- the Windows operating system. That means the company must also contend with the increasingly capable Linux platform. Winter Corp. surveys reveal that Linux, which barely registered in the TopTen program in 2003, is becoming a fixture on the list. The largest Linux data warehouse, a 24.8 TB Oracle-based deployment for Amazon.com, is 27 percent larger than the top-ranked Windows-based deployment.

Just as important, says Winter, Windows and Linux have settled into the same area of the database market. "For servers of a certain scale, the two most popular options are Windows and Linux."

DiDio says Microsoft is doing the right things to combat the Linux threat. The release of SQL Server 2005 Express Edition and deep integration with Visual Studio and BizTalk Server offer a capable bulwark against the emerging threat.

"If you are Microsoft, paranoia when it comes to Linux is a good thing," says DiDio. "You want to shut it down at the fringes. They recognize that they will lose some business to MySQL. They recognize that two of their most powerful competitors in Oracle and IBM have thrown their hats into the Linux ring."

For all that, DiDio says the success of SQL Server 2005 will come down to simple execution.

"At the end of the day, I think 40 percent to 50 percent of Microsoft's installed base will be taking a look at this. A lot will be riding on this first wave. Bugs or early security issues could be devastating."

Michael Desmond is Redmond magazine's editor at large.

Microsoft is also taking steps with Office 12 to make its dominant office productivity suite a premiere BI client platform. Nigel Pendse, an independent, U.K.-based analyst and author of The OLAP Report, who has chronicled the decimation of the OLAP market since Microsoft entered, suggests Microsoft will make progress with Office.

"Microsoft will not instantly dominate the BI front-end market -- but if it enters the market with a reasonably competent, low-priced set of BI capabilities in Office 12, this will start squeezing the client tool vendors' profitability, just as happened with OLAP servers, at least ten of which have disappeared from the market after Microsoft released OLAP Services at the end of 1998," Pendse explains.

Clearly, all this integration is in Microsoft's best financial interest. If every sale of SQL Server can also generate the sale of a Windows Server license, a Microsoft Operations Manager license and a few Office (and Windows desktop) CALs, it stands to be a fiscal bonanza.

Even as Microsoft tightly integrates its products, the company faces a countervailing trend in the growing adoption of XML and Web services. For nearly five years, Microsoft has stumped its .NET vision, touting the simple, plain text interchange of business data via XML. That success now makes proprietary lock-in across products more difficult for Microsoft to achieve.

SQL's Share on Windows
"Microsoft [SQL Server] is the top relational DBMS on the platform -- [the company] had 45 percent of the total share in 2003 and every indication is that [its] position strengthened in 2004. [It was] followed by Oracle and IBM."
-- Carl Olofson, IDC

Key Microsoft executives, like vice president and regular Microsoft TechEd keynoter Paul Flessner, have touted the customer benefits of their "better together" approach of integrating Microsoft servers for several years. In fact, Flessner and others have said customers demand that Microsoft products should seamlessly interoperate. That thinking helped drive the Dynamic Systems Initiative that currently draws Microsoft's products ever tighter.

Jim Vierra, a systems consultant in the New York City metropolitan area, is one customer who sees value in the increased integration of Microsoft products, although he believes the most important aspects of SQL Server 2005 are the extensibility of the Common Language Runtime and the new and enhanced XML storage and querying capabilities.

"From a technical standpoint, SQL Server 2005 provides many new hooks and features that will simplify building complex solutions to age old problems of business information management using a classic data store," Vierra says.

Christopher Bell, a consultant and system architect in Glossop, England, agrees there are user benefits in the integration of the database with the operating system and development tools. "My concern lies in Microsoft's tendency to lock in the customer. I know that this is just how business works and Microsoft is not doing anything that other businesses aren't," Bell says. "What I do have a problem with is the direction in which such integration leads: potential difficulty to integrate a third party product into the mix and a software object of such size and complexity that change becomes more difficult and slower."

"It seems that Microsoft stands alone in not understanding that tight integration of more and more features simply provides a greater vector for system failure, which is not easily fixed," says Bell.

Bell's last concern may be borne out by SQL Server's tardy delivery schedule. The database project code-named "Yukon" has been tentatively scheduled for release every year since 2003.

Integration across the software stack has pushed back on the schedule for years. Companywide initiatives from Trustworthy Computing to WinFS to Springboard have also gotten in the way. (See "Potholes in a Tightly Coupled Roadmap.")

Potholes in a Tightly Coupled Roadmap

Part of the reason SQL Server 2005 took so long to get from code-name "Yukon" to gold code is the tight integration with so much of the rest of the Microsoft software stack. What follows are some of the integration twists and other turns and obstacles that contributed to SQL missing public ship dates in 2003 and 2004 and nearly slipping into 2006.

Trustworthy Computing -- The company-wide security training, security review and code scrub announced in early 2002 pushed back delivery schedules for everything after Windows Server 2003.

SQL Slammer -- A true white knuckle moment for Microsoft. The company was so floored by this January 2003 worm, and customers so damaged and outraged, that executives are still apologizing for it.

Dynamic Systems Initiative -- Requires that Windows Server System products like SQL Server clear a number of integration hurdles involving other Microsoft products.

Springboard -- A Trustworthy Computing-related reassessment of already shipping products for security improvements. The process produced Windows XP SP2 and Windows Server 2003 SP1. SQL Server 2000 was considered for a Springboard service pack.

WinFS -- the SQL Server-based file system for Windows. The on-again, off-again status of this major project seems to have been a distraction in the "Yukon" effort.

Kodiak -- A version of Exchange Server with the signature feature being a SQL Server-based file system. "Kodiak" was eventually killed.

Testing, Testing -- As Microsoft SQL Server gets deployed in more and more mission critical environments, the pressure to ship bug-free code intensifies. The company stepped up with a massive automated testing regimen, complemented by widespread beta and CTP distribution programs. Microsoft IT department has been running SQL Server 2005 in production for a year.

-- S.B.

There are other costs to Microsoft. Despite impressive scalability benchmarks and high-profile reference customers, SQL Server remains just outside the top tier of database scalability on the TPC-C and TPC-H benchmarks. The most likely reason: Windows doesn't run on the most scalable hardware and Microsoft has chosen not to port SQL Server to anything but Windows.

Internal Integration
On the other hand, SQL Server revenues are a bright spot in Microsoft financial reports every quarter, suggesting Microsoft hasn't been as flat-footed as the five-year span between major releases might imply. A closer look at post-RTM SQL Server 2000 shows the database saw the addition of two major functionality areas between releases -- the interesting Notification Services and the colossal Reporting Services.

What's most important about both those add-ons is that they are free with the base price of SQL Server. They join a long tradition of otherwise pricey data management products that Microsoft has chosen to package for free in SQL Server, such as extraction, transformation and load functionality; online analytic processing (OLAP); data mining algorithms; and disaster recovery capabilities

The SQL Server 2005 family includes four variants:
SQL Server 2005 Enterprise Edition
$25,000 per processor
SQL Server 2005 Standard Edition $6,000 per processor
SQL Server 2005 Workgroup Edition $3,899 flat fee (max. two processors)
SQL Server 2005 Express Edition Free

Now users are eager to try out new packaged elements coming in SQL Server 2005. Rob Reimer, working in the United Arab Emirates, sees value in new data mining algorithms that Microsoft is adding to the limited set that came with SQL Server 2000. "The data mining components of SQL Server 2005 will take data mining out of the realm of the esoteric and make it accessible to all, just like SQL Server 2000 reduced the requirement for a full time DBA to run a relational database," Reimer says.

Clive Chinery, a consultant in England, looks forward to database mirroring -- a feature that isn't ready yet, but should be shortly. "All databases have become better as a result of the competition from Microsoft," Vierra contends.

At the very least, databases and associated BI products have become cheaper. Microsoft marketers still use pricing slides versus Oracle and IBM to devastating effect -- showing how the three databases are similarly priced for enterprises until you start adding in OLAP, clustering, database mirroring, and other functionality. Prices of Oracle and IBM start climbing while SQL Server stays flat.

But Oracle and IBM are following Microsoft's lead, with lower prices and more integrated stacks.

"All three are trying to extend their market position by adding such functions as BI integration, complex data type and messaging support, and data quality/integration," notes Michael Gonzales of HandsOn-BI LLC, in a recent report for TDWI. "Although these leading database vendors may not achieve best-in-class for the specific technology categories above, they will offer sufficient functionality. And, when you combine that functionality with their price/bundling leverage and the ability to control and administer all within a single enterprise architecture, the overall value is significant."

Database Market Share
by Revenue, 2004

Database Market Share by Revenue, 2004

Source: IDC

Making 'Good Enough' Cheaper
By continuing its strategy of making each SQL Server sale a Windows sale -- while bundling additional data management components -- Microsoft both helps and hurts its customers.

Across SQL Server's lifespan, Microsoft has driven down the cost of high-quality, high-performance relational database management systems -- a good thing for customers. At the same time, Microsoft has exposed lower-end customers to OLAP, data mining and other formerly niche technologies by bundling such functionality at no additional cost. The tight integration with Visual Studio led to faster development of database-backed applications and the use of Windows as an exclusive platform helped Microsoft make database setup and management far more straightforward for IT generalists. Whether by design or not, Microsoft failed to knock out the highest-end vendors and applications in database markets -- leaving customers the options to graduate to best-of-breed products when they bump into scalability or functionality ceilings with SQL Server's add-on components.

On the other hand, those markets are less innovative. Market-leading vendors like Hyperion Software survived Microsoft's OLAP bundling, but more than ten major vendors have exited the market in the meantime. What's more, the barrier to entry is now high enough that analysts don't expect innovation from startups.

Meanwhile, Microsoft's own monolithic software stack affects the speed at which its own new versions come out. And SQL Server loads exclusively on a server operating system that is a close relative of the Windows client, which due its presence on more than 90 percent of desktops is the most attractive hacker laboratory in the world. In other words, attackers targeting SQL Server start with the best work done by the most talented white and black hats in the business.

Is integration good for you? Like anything in IT, it depends on what's most important in your environment.

Featured