Through a new, efficient algorithm for exact sampling, Microsoft researchers Daniel Tarlow and Tom Minka, along with former Microsoft intern Chris Maddison, address a core problem of statistics and machine learning (ML).

The authors submitted their algorithm at NIPS 2014 where it was picked as one of the two Outstanding Paper Award winners from a field of 1,700 submissions.

Daniel and Tom

“This research makes a very significant advance in the efficiency of sampling, which is a core component of probabilistic modelling and reasoning systems,” said Andrew Blake, Distinguished Scientist and Laboratory Director of Microsoft Research Cambridge.

Tarlow’s hope for the future is these findings will lead to probabilistic reasoning systems that are more powerful and easier to use than current systems. “When we can provide stronger guarantees about the quality of outputs from our inference algorithms, it becomes easier to use these algorithms inside larger systems and to build tools that can be used reliably by non-experts,” he said.

You can read their paper, titled A* Sampling from this site.

ML is a key focus of Microsoft Research and has led to numerous product contributions including Microsoft Office, SQL Server, Xbox One, Cortana speech recognition, and Skype Translator. Additionally, several-state-of-the art ML algorithms from Microsoft Research are offered as part of Microsoft's cloud-based Azure ML platform for predictive analytics.

ML Blog Team

In case you missed it: Channel 9 recently featured a discussion and demos of our preview release of Azure Data Factory.

In this video, Anand Subbaraj introduces us to Azure Data Factory (ADF), a new Azure service that helps data developers and IT professionals easily transform raw data into trusted data assets for their organization at scale.

ADF operates over a range of data services, and supports processing of on-prem SQL Server, Azure SQL Database, Blobs, and Tables using Hive, Pig and C# on HDInsight (Hadoop). With ADF, you can easily create and orchestrate simple, highly available, fault tolerant data analytics pipelines which can be monitored from the Azure Preview Portal.

Organizations all over the world are collecting, processing and gaining insights from more data than ever before – with ADF pipelines you can deliver transformed data from the cloud back to on-premises sources like SQL Server, or keep it in cloud storage; you can take advantage of the seamless connection with Power BI and other applications for the consumption of data assets.

To check out the video, click the image below:

You can click here to get started with ADF today. And, for those of you interested to stay tuned into Microsoft’s world of data - be it relational databases or non, established products or brand new cloud-based advanced analytics services - the Data Exposed section of Channel 9 is a good site for you to bookmark.

ML Blog Team

The range of applications where Machine Learning (ML) can be applied never cease to amaze us.

Recently, a team in Microsoft China which included an intern whose father raises pigs built a phone app to help small-scale pig farmers decide on the types and quantity of pigs that will maximize their profits and minimize market risk, helping them run their farms more efficiently. Their nifty app, which calls into an Azure ML API in the backend, analyzes historical data and makes business predictions of great value for their target customer base. 

Play this video to learn more about Dr. Pig.

This story was on the cover of the Microsoft News Center today - there you can learn more about this wonderful app, including some of the people behind it.  

ML Blog Team

Interested in growing your BI and Big Data skills in 2015? Maybe your new year’s resolution is all about learning something new or taking your analytics knowledge to the next level?

See what BI and Big Data training courses were your peers’ favorites in 2014:

• Faster Insights to Data with Power BI Jumpstart
• From Data to Insight and Impact: The BI Revolution
• Getting Started with Microsoft Azure Machine Learning
• Big Data and Business Analytics Immersion v3.1
• Microsoft Corporate BI & SAP BI Integration Jumpstart 
• Designing BI Solutions with Microsoft SQL Server
• Implementing Data Models & Reports with Microsoft SQL Server
• Excel 2013 Power BI Fundamentals
• Implementing Big Data Analysis

And last but not least, check out the brand new Big Data with the Microsoft Analytics Platform Services.

You can always find training opportunities on the Microsoft Virtual Academy, so check back often.

As one part of the Microsoft Azure for Research project we periodically run RFPs and offer awards that grant access to Azure Machine Learning to researchers, instructors and students. The goal of these awards is to facilitate scholarly and scientific research by enabling researchers and instructors to take advantage of the power and scale of cloud computing and leverage ML for their classroom and advanced analytics needs via shared and collaborative workspaces

The winners of our November 2014 RFP are below. Congratulations to you all! It is quite amazing to see the diversity of winning ideas and the many different places from which they originated.

For those of you interested to apply for an Azure ML award, you can do so by the February 15th deadline for our next RFP, here.

Research Award Winners

Instruction Award Winners

ML Blog Team

Re-post from the Official Microsoft Blog

Our customers generate enormous volumes of data every day including tons of email and documents. Sifting through all that data to find what is relevant to a specific legal or compliance issue can be prohibitively expensive and time consuming.

We acquired Equivio with the goal of helping our customers tackle such challenges, helping them manage and explore large, unstructured data sets and quickly zoom into what is relevant. Learn more by clicking here or on the logo below:

ML Blog Team

The NuGet packages for OData Web API 5.4 beta are now available on the NuGet gallery.

Download this release
You can install or update the NuGet packages for OData Web API 5.4 beta using the Package Manager Console:

PM> Install-Package Microsoft.AspNet.OData -Version 5.4.0-beta -Pre

PM> Install-Package Microsoft.AspNet.WebApi.OData -Version 5.4.0-beta -Pre 

What’s in this release?
This release primarily includes new features for OData (v4 and v3) Web API as summarized below:

• Referential constraint (v4, v3) #37
• Relax flag for version constraint (v4, v3): By default requests with both v3 and v4 max version headers will no longer fail. #191
• DateTime support (v4) #136
• Case-insensitive support (v4) #11
• StoreGeneratedPattern annotation (v3) #189
• ConcurrencyMode annotation and ETag (v3) #190

V4 package has a dependency on ODL 6.9.

Questions and feedback
You can submit questions related to this release, any issues you encounter and feature suggestions for future releases on our GitHub site.

Re-posted from the Official Microsoft Blog

Microsoft has reached an agreement to acquire Revolution Analytics. 

Announcing this earlier today, Joseph Sirosh, Corporate Vice President of Information Management and Machine Learning at Microsoft, said “Revolution Analytics is the leading commercial provider of software and services for R, the world’s most widely used programming language for statistical computing and predictive analytics. We are making this acquisition to help more companies use the power of R and data science to unlock big data insights with advanced analytics.”

You can read the original post here or by clicking the graphic below:  

ML Blog Team

Our Friday announcement about Revolution Analytics generated a lot of interest worldwide with positive coverage in many technology and business publications and on social channels including Hacker News and Twitter. Here’s a small sampling of quotes - and the logos are linked to the corresponding article.

“Though Microsoft already works with R, this represents a new bet on the language, reflecting the company’s wider interest in data science… The move deepens Microsoft’s investments in open source.”

"Revolution Analytics and the R project… are a big deal in the world predictive analytics and machine learning. That’s an emerging market that Microsoft wants to get in on early, while so many other vendors are still pushing yesterday’s technologies or focused on building out infrastructure to store all the data companies want so badly to analyze."

"Revolution Analytics counts financial companies such as American Century Investments and Northern Trust as customers… The R programming language is widely used by statisticians and scientists and has surged in popularity as people have turned to it to manipulate large pools of data."

Finally, you can also read this post by David Smith, Chief Community Officer at Revolution Analytics, who had this to say:

"This is an exciting new chapter for the Revolution Analytics team. We’re excited the work we’ve done with Revolution R will come to a wider audience through Microsoft."

ML Blog Team

Earlier this week, James Phillips, General Manager for Data Experiences at Microsoft, unveiled the new Power BI, a cloud-based business analytics service designed for non-technical business users.

Anyone with a US business email account can now try the preview of the new Power BI for free (the service will become available to international users in the future). With just a browser or a mobile app, you can keep a pulse on your business via “live” operational dashboards.

Catch all the coolness of Power BI in this demo:

The service will transform the “business of business intelligence,” as James put it. You can read James’ complete blog post here.

ML Blog Team

In an effort to provide an extra level of convenience, we are releasing two features that will simplify the effort to ensure the health of your SQL Virtual Machine and your data. These features, Automated Backup and Automated Patching, automate the processes of backing up and patching your SQL Virtual Machines. Incredibly easy to set up, these features require little input to manage. And these will just be the initial services to be automated.

These services will be available to you for configuring SQL VMs in Azure via Portal and PowerShell. Via PowerShell, you will be able to enable these services for new and existing SQL VMs. In the Azure Portal, you will be able to enable these services when provisioning new VMs.

Automated Backup

This service enables you to configure a backup schedule on your SQL Server 2014 Enterprise and Standard Virtual Machines in a very convenient manner while ensuring your data is backed up consistently and safely. Automated Backup is configured to backup all existing and new databases for the default instance of SQL Server. This simplifies the usual process of configuring Managed Backup for new databases and then for each existing database by combining it into one simple automated setup.

This feature is disabled by default, and once it is enabled, requires very little effort to configure. If you do not wish to change the default settings, no work is required beyond enabling the service. If you wish to customize the settings, you can specify the retention period, storage account, and whether you want encryption to be enabled. The retention period, as is standard for Managed Backup, can be anywhere between 1 and 30 days. The storage account defaults to the same storage account as the VM, but can be changed to any other storage account. This provides you with a DR option, allowing you to back up your databases to storage in another datacenter. If you decide to encrypt your backups, an encryption certificate will be generated and saved in the same storage account as the backups. In this scenario, you will also need to enter a password which will be used to protect the encryption certificates used for encrypting and decrypting your backups. This allows you to not worry about your backups beyond the configuration of this feature, and also ensures you can trust that your backups are secure.

You can see a screenshot of what you will see in the Azure Portal here:

Automated Patching

Many customers told us that they would like to move their patching schedules off business hours. This feature enables you to do exactly this – define the maintenance window that would keep your patch installs in the range you have specified.

When you look on the settings available for the Automated Patching you could find you are familiar with those, because they mimic settings available from the Windows Update Agent (service that drives patching of your Windows machine). Settings are simple and powerful at the same time. All that you need to define to make sure patches are applied when you want is: day of the week, start of the maintenance window, and duration of the maintenance window. It relies on the Windows Update and the Microsoft Update infrastructure and installs any update that matches the ‘Important’ category for the machine.

This feature allows you to patch your Azure Virtual Machines in effective and predictable way even when those VMs are not joined to any domain and not controlled by any patching infrastructure.

There are a number of ways how you can configure Automated Patching, but the easiest way is to use new Azure Portal, you can see how the configuration screen can look like on the screenshot below.

SQL Server IaaS Agent

Both features are part of the new component that will be installed on the VM when features are enabled and this component is called SQL Server IaaS Agent. It is built in the form of Azure VM Extension meaning all the Azure VM Extension concepts are applicable making it perfect tool for the management of SQL in Azure VMs on scale. You can push this IaaS Agent to a number of VMs at once, you can configure, and you can remove or disable it as well.

This IaaS Agent moves SQL Server one step closer to be the best application to run in Azure VMs.

Try these features out for yourself at https://portal.azure.com.

For further details, here is the documentation page for these features.

In August, we announced the SQL Server AlwaysOn Offering in the Microsoft Azure Portal Gallery. This offering fully automates the configuration of a highly available SQL Server deployment on Azure Infrastructure Services using AlwaysOn Availability Groups.

Now, we have updated this offering with some exciting improvements. Namely, we have added support for using existing domains, and we have optimized the time it takes to deploy, so you save even more time than before!

AlwaysOn Availability Groups in Azure

SQL Server AlwaysOn technology provides the high availability capabilities needed for mission critical applications.  One of the main challenges of this technology is that it requires a complex and time-consuming setup. With the new SQL Server AlwaysOn Template in the Azure Portal, this process is greatly simplified, freeing up your valuable time and resources. After entering the desired settings in the Portal, the setup is automatically completed for you. With this setup you get an Availability Group with two SQL VMs, a listener configured to point to the current primary, a file share witness, failover cluster, and two domain controller VMs for a new or existing domain.

Existing Windows Domain

With this update, you can select between having a new domain created for this configuration and utilizing an existing domain you have pre-configured with all your specific requirements. You can use the new domain option to have a domain fully created and set up for you. This is a good option if you do not have very specific domain requirements, or do not have an existing domain you wish to utilize. You can use the existing domain option if you have very specific requirements or prefer to reuse a pre-configured domain from on-premises or in Azure. With this option selected, the SQL primary, secondary, and file share witness will be successfully added to your existing domain.

To use an existing domain, first select the correct existing Virtual Network for that domain, then select the existing domain with the user credentials, as shown in the screenshot below.

Execution Time Cut in Half

Going through the manual setup of an AlwaysOn Availability Group can take anywhere from 2-6 hours, depending on your level of expertise on the technology. When this feature was initially released, it decreased the amount of work to set this up to 1 minute, and had the configuration completely deployed after about 1.5 hours. Now we have decreased that even further, to around 45 minutes. This saves even more valuable time and allows you that much faster access to highly available SQL VMs to use for your critical business applications.

By Tiffany Wissner, Sr. Director for Data Platform Marketing

Today we are announcing general availability for the latest update to Azure SQL Database, introduction of new SQL Database security features, more automation for SQL Server in Azure Virtual Machines and SQL Server on G-Series VMs.  Take a closer look at these exciting new improvements.

1.       New update to Azure SQL Database Generally Available in Europe

Generally available today in Europe, the latest version of SQL Database introduces near-complete SQL Server engine compatibility, greater support for larger databases, and expanded Premium performance. Internal tests on over 600 million rows of data show Premium query performance improvements of around 5x in the new preview relative to today’s Premium SQL Database and up to 100x when applying the In-memory columnstore technology. 

General availability will continue across United States regions on February 9, 2015 with rollout to most datacenters worldwide by March 1, 2015. General availability pricing will take effect for databases on V12 servers worldwide on April 1, 2015.

“As a company committed to maintaining the highest innovation standards for our global clients, we’re always eager to test the latest features,“ said John Schlesinger, Chief Enterprise Architect at Temenos. “So previewing the latest version of SQL Database was a no-brainer for us. After running both a benchmark and some close-of-business workloads, which are required by our regulated banking customers, we saw significant performance gains including a doubling of throughput for large blob operations, which are essential for our customer’s reporting needs.” 

2.       New Security Features for Azure SQL Database

Today also marks the introduction of a suite of security features coming to the latest version of SQL Database; Row-Level Security, Dynamic Data Masking, and Transparent Data Encryption. These security features will join the existing Auditing feature to help customers further protect their cloud data and help further meet corporate and industry compliance policies. Available in public preview today across all of the new service tiers, customers can implement Row-level Security on databases to enable implementation of fine-grained access control over rows in a database table for greater control over which users can access which data.

Coming soon, SQL Database will also preview Dynamic Data Masking which is a policy-based security feature that helps limit the exposure of data in a database by returning masked data to non-privileged users who run queries over designated database fields, like credit card numbers, without changing data on the database. Finally, we are excited to announce that Transparent Data Encryption is coming to SQL Database V12 databases for encryption at rest. As data security is at the top of mind for customers building new applications in the cloud, these new security features will be available the Basic, Standard and Premium service tiers. 

3.       More simplified availability, setup, backup, and patching for SQL Server in an Azure VM

SQL Server AlwaysOn technology provides both the high availability and disaster recovery capabilities needed for mission critical applications.  One of the challenges is getting the HA environment setup, as it’s not trivial.  Now with new auto HA setup capabilities using the AlwaysOn Portal Template added for SQL Server in Azure VMs, this really becomes a simpler task, freeing up your valuable time and resources to focus on other business priorities.  This automated setup provides listener configuration and provisions AlwaysOn VM cluster to meet HA and DR requirements.  This new capability applies for hybrid scenarios where you might be setting up failover for an on-premises SQL Server workload or a cloud only SQL Server workload when you want to setup failover from one Azure region to another.

Backups for data security are easier now as well with the ability to automate full SQL Server backups from an Azure VM to Azure Storage.  Additionally, SQL Server patches delivered through Windows Update also get better with new auto patching capability that gives you more granular control over the windows update scheduler for predictable timing of updates. Monitoring and managing SQL Server instances running in Azure VMs gets better as well with the ability to view and manage SQL Server alerts directly through the Azure Portal.

The new suite of new capabilities along with previously released Azure VM capabilities helps make running large enterprise SQL Server workloads in Azure VMs more efficient than ever before.

4.       SQL Server on Massive G-Series VMs

The new G-Series VMs are ideal for large SQL Server OLTP workloads, especially combined with SQL Server 2014 in-memory OLTP technology, as the new Azure G-Series VMs offer up to 32vCPUs, 448GB of memory, and 6.59TB of local SSD.  Combine this with the in-memory OLTP technology built-in to SQL Server 2014, you can now maximize the performance of these large VMs by taking advantage of unique parallel processing in-memory architecture that removes database dead locks while ensuring 100% durability.  This means not only can you get x factor transactional performance gains, but you can also gain x factor improvement in concurrency by taking full advantage of 32vCPUs. 

Great options in Azure for your SQL Server enterprise workloads

Whether you are looking to run your SQL Server workload in an Azure virtual machine or via the SQL Database managed service, there’s no better time than now to move your enterprise workloads to the cloud or build new applications with Microsoft Azure.

If you have an ecosystem of IT resources who can continue to manage and maintain your application in the cloud, SQL Server on an Azure Virtual Machine is the ideal option—now with even more built-in productivity, power and scale. If you don’t have an ecosystem of IT resources or don’t want them maintaining and patching every database in your portfolio, the greater SQL Server compatibility, predictable Premium performance, built-in 99.99% availability and upcoming suite of security features make Azure SQL Database an ideal destination.

We’re excited to share these ongoing improvements of our SQL Server cloud offerings with you; helping make SQL Server on an Azure VM and Azure SQL Database two great migration or deployment targets for your enterprise-grade SQL Server workloads.

If you haven’t already, start a free trial on SQL Server in Virtual Machines or Azure SQL Database today!

I'm so excited to announce that we are deploying Row-Level Security, a programmability feature to ease the writing of business security logic in the database, to Azure SQL Database. Coming to a region near you as the deployment propagates around the world, it will be available in all V12 server once deployment completes. See the main SQL Server team blog for more details. Technical details should start showing up on MSDN today as those sites are updated.

The NuGet packages for OData Web API 5.4 RC are now available on the NuGet gallery.

Download this release

You can install or update the NuGet packages for OData Web API 5.4 RC using the Package Manager Console:

PM> Install-Package Microsoft.AspNet.OData -Version 5.4.0-rc -Pre

PM> Install-Package Microsoft.AspNet.WebApi.OData -Version 5.4.0-rc -Pre

What’s in this release?

This release primarily includes new features for OData (v4 and v3) Web API as summarized below:

• Referential constraint (v4, v3) #37
• Relax flag for version constraint (v4, v3): By default requests with both v3 and v4 max version headers will no longer fail. #191
• DateTime support (v4) #136
• Case-insensitive support (v4) #11
• StoreGeneratedPattern annotation (v3) #189
• ConcurrencyMode annotation and ETag (v3) #190
• Bug fixes: CodePlex&GitHub

V4 package has a dependency on ODataLib 6.9.

Questions and feedback

You can submit questions related to this release, any issues you encounter and feature suggestions for future releases on our GitHub site.

Row-Level Security Preview

Yesterday we announcedexciting news for Power BI– a cloud-based business analytics service (software-as-a-service) for non-technical business users.  The preview introduces a number of new Power BI capabilities including dashboards, new visualizations, support for popular software-as-a-service applications, a native iPad app and live “hybrid” connectivity to on-premises SQL Server Analysis Services tabular models. With just a browser – any browser – or a Power BI mobile app, customers can keep a pulse on their business via live operational dashboards. They can explore their business data, through interactive visual reports, and enrich it with additional data sources.

How does it work with SQL Server?
To interact with SQL Server data in Power BI, connect to SSAS server via the ‘Get Data’ menu. From there, you can connect to a model and run queries for visualizations based on that model. Before your users can connect to an SSAS model, an administrator must configure a Power BI Analysis Services connector.

To learn more about the Power BI preview, watch as Michael Tejedor gives Jeremy Chapman from Office Mechanics a first look at what’s new.