News

Microsoft Launches Preview of Azure Databricks

Microsoft this week unveiled a new service that integrates its Azure platform with a globally distributed streaming analytics solution.

Now available in preview, Azure Databricks is designed to help app users and developers take advantage of machine learning, graph processing and AI-based applications. It was one of many announcements made by Microsoft during the opening keynote of its annual Connect() developer conference, which kicked off Wednesday in New York City.

Azure Databricks will enable organizations to build modern data warehouses that support self-service analytics and machine learning using all data types in a secure and compliant architecture, according to Scott Guthrie, Microsoft's executive vice president of Cloud and Enterprise, speaking at the the Connect() keynote.

Databricks is the creator and steward of Apache Spark. Azure Databricks is effectively a first-party Spark-as-a-Service platform for Azure. "It allows you to quickly launch and scale up the Spark service inside the cloud on Azure," Guthrie said. "It includes an incredibly rich, interactive workspace that makes it easy to build Spark-based workflows, and it integrates deeply across our other Azure services."

Those services include Azure SQL Data Warehouse, Azure Storage, Azure Cosmos DB, Azure Active Directory, Power BI and Azure Machine Learning, Guthrie said. It also provides integration with Azure Data Lake stores, Azure Blob storage and Azure Event Hub. "It's an incredibly easy way to integrate Spark deeply across your apps and drive richer intelligence from it," he said.

Databricks customers have been pushing the company to build its Spark platform as a native Azure service, said Ali Ghodsi, the company's co-founder and CEO, who joined Guthrie on stage. "We've been hearing overwhelming demand from our customer base that they want the security, they want the compliance and they want the scalability of Azure," Ghodsi said. "We think it can make AI and big data much simpler."

In addition to integrating with the various Azure services, Azure Databricks is designed to let those who want to create new data models to do so. According to Databricks, a user can target data regardless of size or create projects with various analytics services, including Power BI, SQL Server, Streaming, MLlib and Graph.

"Once you manage data at scale in the cloud, you open up massive possibilities for predictive analytics, AI, and real-time applications," according to a technical overview of the Azure Databricks service. "Over the past five years, the platform of choice for building these applications has been Apache Spark. With a massive community at thousands of enterprises worldwide, Spark makes it possible to run powerful analytics algorithms at scale and in real time to drive business insights."

However, deploying, managing and securing Spark at scale has remained a challenge, which Databricks believes will make the Azure service compelling.

Internally, Databricks is using Azure Container Services to run the Azure Databricks control plane and data planes using containers, according to the company's technical primer. It's also using accelerated networking services to improve performance on the latest Azure hardware specs.  

About the Author

Jeffrey Schwartz is editor of Redmond magazine and also covers cloud computing for Virtualization Review's Cloud Report. In addition, he writes the Channeling the Cloud column for Redmond Channel Partner. Follow him on Twitter @JeffreySchwartz.

Featured

  • Closeup of the new Copilot keyboard key

    Microsoft Updates Copilot To Add Context-Sensitive Agents to Teams, SharePoint

    Microsoft has rolled out a new public preview for collaborative "always on" agents in Microsoft 365 Copilot, bringing enhanced, context-aware tools into Teams channels, meetings, SharePoint sites, Planner workstreams and Viva Engage communities.

  • Windows 365 Cloud Apps Now Available for Public Preview

    Microsoft announced this week that Windows 365 Cloud Apps are now available for public preview. This aims to allow IT administrators to stream individual Windows applications from the cloud, removing the need to assign Cloud PCs to every user.

  • Report: Security Initiatives Can't Keep Pace with Cloud, AI Boom

    The increasingly fast adoption of hybrid, multicloud, and AI systems is easily outgrowing existing security measures, according to a recent global survey by the Cloud Security Alliance (CSA) and exposure management firm Tenable.

  • World Map Image

    Microsoft Taps Nebius in $17B AI Infrastructure Deal To Alleviate Cloud Strain

    Microsoft has signed a five-year, $17.4 billion agreement with Amsterdam-based Nebius Group to expand its AI computing capabilities through third-party GPU infrastructure.