Microsoft Taps SQL Server PDW To Boost Security Forensics

Microsoft published details this month about how it used its own Parallel Data Warehouse (PDW) edition of SQL Server 2008 R2 to improve the performance of its company-wide security monitoring system.

The PDW edition is scheduled for release on HP's hardware appliance next month, which Microsoft announced last week. PDW features massive parallel processing capacity, which represents a step up in performance over other relational database management systems in the SQL Server 2008 R2 product line. With PDW, each node on the machine runs its own instance of SQL Server and has dedicated memory, storage and CPU power.

Microsoft began testing the PDW system during the Technology Adoption Program (TAP) stage to see if it could improve the performance of its proprietary network security monitoring solution, which is called Information Security Consolidated Event (ICE) 4.0.

ICE is used for Microsoft's forensic security analysis and research. However, Microsoft also deploys it as part of its efforts to secure its worldwide networks, consisting of more than 250,000 computers across more than 100 countries, according to a TechNet library article.

The aim of the TAP deployment was to scale ICE's capacity upward from its existing 60 terabytes of server log data. Microsoft also wanted to extend data retention times of the ICE data and improve overall system performance.

Microsoft claims in the TechNet article that it achieved all of those goals with the PDW deployment. Under the TAP program testing, the log data retention time was extended from 60 days to 90 days. The extract, transform and load handling increased from 1 terabyte per day to 4 terabytes per day. Complex queries ran an average of 20 times faster than they ran before Microsoft used PDW, the articles states. Microsoft had previously run its ICE solution on the 64-bit version of SQL Server 2008 R2.

More details about the Microsoft's internal PDW case study can be found in an October video presentation by Pete Loveless, a senior program manager for the Microsoft trustworthy computing group, who describes the TAP deployment. He said that query times had improved with PDW, even during peak loads. One query that took over one hour to complete with the earlier system took just four minutes with the PDW deployment, Loveless said.

About the Author

Kurt Mackie is senior news producer for 1105 Media's Converge360 group.


  • Touting Azure for Operators, Microsoft Joins SDN Standards Group

    As part of its Azure for Operators program, Microsoft this week joined a nonprofit standards association that focuses on SDN technologies used by enterprises and service providers.

  • 2020 Microsoft Conference Calendar: For Partners, IT Pros and Developers

    Here's your guide to all the IT training sessions, partner meet-ups and annual Microsoft conferences you won't want to miss. (Now updated with COVID-19-related event changes.)

  • Pilot Begins of Microsoft Teams-Salesforce CRM Integration

    A new capability that lets Microsoft Teams users access information from the customer relationship management (CRM) platform debuted this week.

  • The 2020 Microsoft Product Roadmap

    From the next major update to Windows 10 to the next generations of .NET and PowerShell, here's what's on tap from Microsoft this year.