Developments in AzureR

by Hong Ooi, senior data scientist, Microsoft Azure

The AzureR packages have now been on CRAN for a couple of months, so I thought I'd provide an update on developments in the works.

First, AAD authentication has been moved into a new package, AzureAuth, so that people who just want OAuth tokens can get it without any other baggage. This has many new features:

• Supports both AAD v1.0 and v2.0
• Tokens are cached in a user-specific directory using the rappdirs package, typically c:users<username>localAzureR on Windows and ~/.local/share/AzureR on Linux
• Supports 4 authentication methods: authorization_code, device_code, client_credentials and resource_grant
• Supports logging in with a username or with a certificate

In the longer term, the hope is for AzureAuth to be something like the R equivalent of the ADAL client libraries. Things to add include dSTS, federated logins, and more.

AzureRMR 2.0 has a new login framework and no longer requires you to create a service principal (although you can still provide your own SP if desired). Running create_azure_login() logs you into Azure interactively, and caches your credentials; in subsequent sessions, run get_azure_login() to login without having to reauthenticate.

AzureStor 2.0 has several new features mostly for more efficient uploading and downloading:

• Parallel file transfers, using a pool of background processes. This greatly improves the transfer speed when working with lots of small files.
• Transfer files to or from a local connection. This lets you transfer in-memory R objects without having to create a temporary file first.
• Experimental interface to AzCopy version 10. This lets you do essentially anything that AzCopy can do, from within R. (Note: AzCopy 10 is quite a different beast to AzCopy 8.x.)
• A new framework of generic methods, to organise all the various storage-type-specific functions.

A new AzureKusto package is in the works, for working with Kusto/Azure Data Explorer. This includes:

• All basic functionality including querying, engine management commands, and ingesting
• A dplyr interface written by Alex Kyllo.

AzureStor 2.0 is now on CRAN, and the others should also be there over the next few weeks. As always, if you run into any problems using the packages, feel free to contact me.