Azure SDK August 2019 preview and a dive into consistency

The second previews of Azure SDKs which follow the latest Azure API Guidelines and Patterns are now available (.Net, Java, JavaScript, Python). These previews contain bug fixes, new features, and additional work towards guidelines adherence.

What’s New

The SDKs have many new features, bug fixes, and improvements. Some of the new features are below, but please read the release notes linked above and changelogs for details.

• Storage Libraries for Java now include Files and Queues support.
• Storage Libraries for Python have added Async versions of the APIs for Files, Queues, and Blobs.
• Event Hubs libraries across languages have expanded support for sending multiple messages in a single call by adding the ability to create a batch avoiding the error scenario where a call exceeds size limits and giving batch size control to developers with bandwidth concerns.
• Event Hubs libraries across languages have introduced a new model for consuming events via the EventProcessor class which simplifies the process of checkpointing today and will handle load balancing across partitions in upcoming previews.

Diving deeper into the guidelines: consistency

These Azure SDKs represent a cross-organizational effort to provide an ergonomic experience to every developer using every platform and as mentioned in the previous blog post, developer feedback helped define the following set of principles:

• Idiomatic
• Consistent
• Approachable
• Diagnosable
• Compatible

Today we will deep dive into consistency.

Consistent

Feedback from developers and user studies have shown that APIs which are consistent are generally easier to learn and remember. In order to guide SDKs from Azure to be consistent, the guidelines contain the consistency principle:

• Client libraries should be consistent within the language, consistent with the service and consistent between all target languages. In cases of conflict, consistency within the language is the highest priority and consistency between all target languages is the lowest priority.
• Service-agnostic concepts such as logging, HTTP communication, and error handling should be consistent. The developer should not have to relearn service-agnostic concepts as they move between client libraries.
• Consistency of terminology between the client library and the service is a good thing that aids in diagnosability.
• All differences between the service and client library must have a good, articulated reason for existing, rooted in idiomatic usage.
• The Azure SDK for each target language feels like a single product developed by a single team.
• There should be feature parity across target languages. This is more important than feature parity with the service.

Let’s look closer at the second bullet point, “Service-agnostic concepts such as logging, HTTP communication, and error handling should be consistent.” Developers pointed out APIs that worked nicely on their own, but weren’t always perfectly consistent with each other. For example:

Blob storage used a skip/take style of paging, while returning a sync iterator as the result set:

let marker = undefined;
do {
   const listBlobsResponse = await containerURL.listBlobFlatSegment(
     Aborter.none,
     marker
   );

  marker = listBlobsResponse.nextMarker;
   for (const blob of listBlobsResponse.segment.blobItems) {
     console.log(`Blob: ${blob.name}`);
   }
} while (marker);

Cosmos used an async iterator to return results:

for await (const results of this.container.items.query(querySpec).getAsyncIterator()){
         console.log(results.result)
      }

Event Hubs used a ‘take’ style call that returned an array of results of a specified size:

const myEvents = await client.receiveBatch("my-partitionId", 10);

While using all three of these services together, developers indicated they had to work to remember more or refresh their memory by reviewing code samples.

The Consistency SDK Guideline

The JavaScript guidelines specify how to handle this situation in the section Modern and Idiomatic JavaScript:

☑️ YOU SHOULD use async functions for implementing asynchronous library APIs.

If you need to support ES5 and are concerned with library size, use async when combining asynchronous code with control flow constructs. Use promises for simpler code flows.  async adds code bloat (especially when targeting ES5) when transpiled.

☑️ DO use Iterators and Async Iterators for sequences and streams of all sorts.

Both iterators and async iterators are built into JavaScript and easy to consume. Other streaming interfaces (such as node streams) may be used where appropriate as long as they're idiomatic.

In a nutshell, it says when there is an asynchronous call that is a sequence (AKA list), Async Iterators are preferred.

In practice, this is how that principle is applied in the latest Azure SDK Libraries for Storage, Cosmos, and Event Hubs.

Storage, using an async iterator to list blobs:
for await (const blob of containerClient.listBlobsFlat()) {
   console.log(`Blob: ${blob.name}`);
}

Cosmos, still using async iterators to list items:
for await (const resources of resources.
                                 container.
                                 items.
                                 readAll({ maxItemCount: 20 }).
                                 getAsyncIterator()) {
     console.log(resources.doc.id)
}

Event Hubs – now using an async iterator to process events:
for await (const events of consumer.getEventIterator()){
     console.log(`${events}`)
   }

As you can see, a service-agnostic concept—in this case paging—has been standardized across all three services.

Feedback

If you have feedback on consistency or think you’ve found a bug after trying the August 2019 Preview (.Net, Java, JavaScript, Python), please file an Issue or pull request on GitHub (guidelines, .Net, Java, JavaScript, Python), or reach out to @AzureSDK on Twitter. We welcome contributions to these guidelines and libraries!