Real-time data streaming with Event Hubs on Azure Government

Real-time data streaming with Event Hubs on Azure Government


In this video we will look at the capabilities of Azure Event Hubs and a simple demo integrating Azure Functions Cosmos database and Power BI. Azure Event Hubs is the data streaming platform that can receive and process millions of events per second it can be widely used in a variety of iot applications telemetry data and any other big data streaming need may have setting up …an event hub in the Azure portals quick and easy here we can see we have already navigated …Government portal click on Event Hubs. Here is an event hub name space that we have already deployed …navigate entities click on Event Hubs. We can see all of the Event Hubs…have that we have under this name space to create a new one you click on this. We can fill out the name modify the message retention rate as… well as a number of partitions to distribute your messages now let’s look at a real-world …let’s say we have traffic booths in various locations gathering data such as the number an average speed of passing cars we want to send this data over to our event hub here we had a simple console application that will send messages directly to our event …we’ve navigated to the command prompt so we can run …the console location that we were just looking at here we can see …run the help command and we can see the different parameters we need to fill out in order to send a message so let’s send a message. We have set the location to houston the speed to fifty miles per hour and the number of cars to five hundred. We can see that we have successfully set the message and it continues to send a message on a loop to the…hub. We have just sent messages to eventhub now we need to process those messages as events. Here we have navigated back to the …Azure Government portal and…looking at our function app that has already been deployed. Here is our traffic sensor function that will be triggered when something is written to our event The function will then write the events to a streaming Power BI …point as well as to a Cosmos Database here we have navigated…Cosmos Database center function of has been writing the messages to click on data explorer and navigate to the traffic messages collection. Here we can see the message that we have just written to the event hub …which has triggered our function app to write to the Cosmos Database. Power BI…is a powerful tool that will allow us to visualize our data we can build reports and dashboard from live streaming data in order to do that we need to create a streaming data set here will click on the streaming data set choose api. Here we have the name for streaming data set as …the data values will be sent to our stream. Now let’s click create. Our streaming data set was successfully created Now we can grab are streaming endpoint…start sending data. Now we want our function app to write to the streaming endpoint…that we have just created here we have navigated back to our function up in the Azure portal. We navigate the application settings we can see that there’s a setting called our by Here we can paste…in the url of the streaming endpoint that we’ve just created and click save. Let’s create a new Power BI dashboard so that we can visualize our streaming data set. We have created a brand new Power BI dashboard now let’s create a streaming tile that we can see our data visualizations we click at tile on this top bar and we can choose the customs streaming data set. Then we can choose a streaming data set that we created before. Now we can choose the values that we want for clustered column chart. We have filled in these values and now we want to set the time window to display let’s set the last ten seconds. This will display the data that has been sent to the streaming endpoint in the last ten seconds. So, let’s give this a title. Here we have the streaming …we have just created which is currently empty because we are not sending messages to our …after we created our streaming data set if we look above …ask a question about your data this is the Power BI …q and a functionality this allows users to type in data points that they want in…visualizations and Power BI takes care of the rest… So, let’s say we want to show sensor value by location. Sensor id. For the last ten seconds. Here we can see that Power BI has created a visualization based on what I’ve typed in q and a …here we are just change this to a column chart currently this graph is showing the sum of the values that have been sent to the event help. Let’s say we want to just see the average. And, let’s also filter by sensor type here we can choose a center type to filter by let’s choose…number of cars. Now we have created a visual that shows the average number of cars in the last ten seconds let’s pin this visual to our dashboard. When we go back to our dashboard we can see the number of cars visual that we have just created I’ve also created this average speed visual which is just same data filtered by average speed? Now we’re going to see everything in action here we are open up our console application to send a few messages …set the location to ithaca which we currently do not have in our visualizations. We have now successfully sending messages now let’s look at our visualization we can see that if occur has appeared in our visualizations. Now let’s stop these messages. Now I’m going to change the number of cars that we can see that graph changing in front of our eyes. We’re going to put these two side-by-side to that we can see the visualization. Now we’re going to change the average speed. We take a look at this graph we can see that the average speed is also increasing now let’s say we have two instances of our console app running one with the locations are to ithaca and another with the location set to baltimore. We started sending messages for both we could see that immediately …visualizations are reacting to the messages that were sending for both locations. Here is another our by visualization that I’ve created from our streaming data set it is showing the number of cars by location on a us map. So, let’s see this data side-by-side. Here we’re going to send messages but the location set to houston. Here we can see that the values for houston are continuing to increase. In this video we have walked through a simple scenario where we are sending sensor traffic data to …event hub which then …is a function that will write the data to Cosmos Database as was to Power BI…streaming endpoint we have then walk through the different data visualizations you can set up with Power BI I highly encourage that you navigate the Azure Government event hub sample where the code for the console application as well as the function app that we have demoed lives…we hope this video has shown you how easy it is to get started with Azure Event Hubs …today.

Leave a Reply

Your email address will not be published. Required fields are marked *