by Le Zhang (Data Scientist, Microsoft) and Graham Williams (Director of Data Science, Microsoft)
Microsoft's Cognitive Toolkit (better known as CNTK) is a commercial-grade and open-source framework for deep learning tasks. At present CNTK does not have a native R interface but can be accessed through Keras, a high-level API which wraps various deep learning backends including CNTK, TensorFlow, and Theano, for the convenience of modularizing deep neural network construction. The latest version of CNTK (2.1) supports Keras. The RStudio team has developed an R interface for Keras making it possible to run different deep learning backends, including CNTK, from within an R session.
This tutorial illustrates how to simply and quickly spin up a Ubuntu-based Azure Data Science Virtual Machine (DSVM) and to configure a Keras and CNTK environment. An Azure DSVM is a curated virtual machine image coming with an extensive collection of pre-installed open source data science tools. The Keras R package can be readily setup up on the DSVM so as to experience the fun of deep learning.
The deployment of a DSVM is also largely simplified through a few R commands from a local R session (running on your own laptop), thanks to the AzureSMR and AzureDSVM packages for R. With an Azure subscription (visit Microsoft for a free trial subscription) and an initial setup of the Azure Active Directory App for authority to access and manage Azure resources, data scientists can use R to manage and operate selected Azure resources, including to easily interact with and use Azure DSVMs for data analytical jobs. A deploy-compute-destroy cycle is now trivial to orchestrate within R.
The following code snippets create an Azure resource group (a logical collection of related resources) and a Ubuntu DSVM of a specified size within the new resource group:
library(AzureSMR) library(AzureDSVM) # Authentication. asc <- createAzureContext(tenantID = "<tenant_id>", clientID = "<client_id>", authKey = "<authentication_key>") azureAuthenticate(asc) # Create a resource group. azureCreateResourceGroup(asc, location = "southeastasia", resourceGroup = "dsvmrg") # Deploy a DSVM with specifications. deployDSVM(asc, resource.group = "dsvmrg", size = "Standard_D4_v2", location = "southeastasia", hostname = "mydsvm", username = "myname", authen = "Password", password = "MyDSVM_123", mode = "Sync")
Once this DSVM is deployed it will cost approximately $0.25 per hour (pricing based on size and location). The DSVM can be powered down as required to reduce costs.
Note that the N-series VMs on Azure now include GPU devices. If a DSVM instance is deployed or resized to the N-series, Keras and CNTK will automatically activate GPU-based capabilities to accelerate model training.
Installation and configuration of Keras can be manually performed after a successful deployment of the DSVM. This post-deployment installation process can be accelerated via Azure Virtual Machine Extensions, which is also functionally available in AzureDSVM:
fileurl <- paste0("https://github.com/yueguoguo/Azure-R-Interface", "/blob/master/demos/demo-5/script.sh") addExtensionDSVM(asc, location = "southeastasia", resource.group = "dsvmrg", hostname = "mydsvm", os = "Ubuntu", fileurl = fileurl, command = "sudo sh script.sh")
The fileurl and command arguments point to the URL of a script file to download to the DSVM and the command to execute that script on the DSVM, respectively. A sample script for installing and configuring Keras and its R interface in Ubuntu Linux DSVM can be found here.
Once the R script has been successfully run the remote DSVM can be accessed via RStudio server or through a remote desktop (e.g., X2Go). Try the following within R on the DSVM to see whether the Keras R interface is installed and CNTK is configured as its backend:
library(keras) backend()
The output in the console should be "Using CNTK backend" indicating a successful set-up. The Keras R interface provides a set of examples to get started. More information can be found here. Note it is also feasible to switch the backend from CNTK to others such as Tensorflow or Theano, by following the instructions here.
A case study for Solar power forecasting (reproducing CNTK tutorial 106 B) is available here. It illustrates training a time series forecasting model by using Long Short-Term Memory (LSTM) for predicting solar power generation. The LSTM layer basically captures patterns and long-term dependencies in the historical time series data of solar power readings, to predict the maximum value of total power generation on a specific day. The following chart compares the prediction with the true data.
The tutorial's model is a simplified version for ease of repeatability but it can be further improved if epoch size is increased and network topology is more sophisticated.
Have fun with Keras + CNTK in R!