We are excited to announce that live data local testing is now available for public preview in Azure Stream Analytics Visual Studio tools. Have you ever thought of being able to test Azure Stream Analytics queries logic with live data without running in the cloud? Are you excited by the possibility of not having to wait for your queries to deploy and other round-trip delays? Well, this enhancement lets you test your queries locally while using live data streams from cloud sources such as Azure Event Hubs, IoT Hub or Blob storage. You can even use all the Azure Stream Analytics time policies in the local Visual Studio environment. Being able to start and stop queries in seconds and local debugging significantly improves development productivity by saving precious time on the inner loop of query logic testing.
Live data local testing experience
The new local testing runtime can read live streaming data from the cloud or from a local static file. It works the same as the cloud runtime Azure Stream Analytics and therefore supports the same time policies needed for many testing scenarios. The query runs in a simulated environment suitable for a single server development environment and should only be used for query logic testing purposes. It is not suitable for performance/scalability and availability testing.
Key benefits
Query behavior consistency
For any given input, the queries running locally and in the cloud will output the same results. Live data testing in Visual Studio includes full support for time polices and machine learning functions.
Much shorter testing cycles
Compared to running in the cloud, it takes only seconds to start a round of query testing. It enables you to test your query directly in Visual Studio without having to switch environments. You can understand the shape of the data coming to easily adjust the query and see immediately the result on the live data.
Viewing job metrics and error messages
Job metrics such as input data size, output data size, and more are available during local execution. Error messages will also be displayed in testing and debugging.
Fluid experience
- The query runs locally but connects to input data cloud sources directly. No need to sample data to local first.
- Query results are immediately displayed in table format in Visual Studio.
Local testing options
In addition to the current local run capability, with this major improvement, we now support all the local testing options listed below.
|
Input |
Output |
Job Type |
|
Local static data |
Local static data |
Cloud/Edge |
|
Live input data |
Local static data |
Cloud |
|
Live input data |
Live output data (Power BI and ADLS are not supported at this moment) |
Cloud |
Local testing vs. cloud run caveats
This local testing feature should only be used for functional testing purposes. It does not replace the performance and scalability tests you would do in the cloud. It should not be used for production purposes since running in a local environment doesn’t guarantee any service SLA.
While on your machine you can only rely on local resources, when you deploy to the cloud we can scale out to multiple nodes. Additionally, cloud deployment ensures checkpointing, upgrades, and more. It also provides all the infrastructure to run your job 24/7.
The following table shows the comparison between running a query in a local environment versus running it in the cloud. Please note, that only the cloud input options support time policies and machine learning functions, the local input doesn’t support these.
|
Query logic |
JavaScript UDF/UDA |
Machine Learning function |
Time policies |
Execution Environment |
Resource Allocation |
Service SLA |
Scalability |
|
|
Local testing with local input |
Yes |
Yes |
No |
No |
Local simulated, |
No CPU/memory commitment |
No |
No |
|
Local testing with live input |
Yes |
Yes |
Yes |
Yes |
Local simulated |
No CPU/memory commitment |
No |
No |
|
Running in the cloud |
Yes |
Yes |
Yes |
Yes |
Cloud |
By SU |
Yes |
Yes |
Try it now
This feature is amazingly easy to use. Just choose the input/output type you want in the script editor and click “Run locally”. Follow this guide on Testing live data locally using Azure Stream Analytics tools for Visual Studio to try it!
Providing feedback and ideas
The Azure Stream Analytics team is highly committed to listening to your feedback. We welcome you to join the conversation and make your voice heard via our UserVoice. For tools feedback, you can also reach out to ASAToolsFeedback@microsoft.com.
Did you know we have more than ten new features in public preview? Sign up for our preview programs to try them out. Also, follow us @AzureStreaming to stay updated on the latest features.