Azure 101: Monitoring & Logging Part 1
Originally I was going to have my next Azure 101 post be about Networking as that is another set of services that every customer needs to learn about no matter what type of application or system you are planning on deploying. However, more important is the type of monitoring and logging that you may want to implement.
In some cases, you might already have specific tools in place that you are already familiar with and of course the question would be "Can and/or Should those tools be implemented or integrated with Azure services?" For customers though that are modernizing and developing for the Cloud and specifically for Azure, it is going to be important to know what information is available from different Azure services and how can that information be ingested and visualized from a reporting and alerting perspective.
So to stick with the Azure 101 series of blog posts, let's take a video look at all of the 1st-party services that are available in Azure that would fall under the topic of monitoring and logging. Many of these services and features are available for free to all customers. This will not be a deep discussion about how you should build your specific monitoring and logging strategy, but more just a walk-through of each of the available services.
To start off, let's list what the different feature or services are that are available within Azure:
- Azure Monitor
- Activity Log
- Diagnostics
- Metrics
- Network Watcher
- Alerting
- Application Insights
- Log Analytics
- Azure Advisor
- Azure Security Center
Azure Monitor
I am sure that you noticed above that there are a number of sub-bullets underneath of the Azure Monitor topic and that is because if you go to the Azure Monitor service within the Azure Portal, you will see a large list of available services or features underneath. So each of the sub-bullets above correspond to a different feature that is available within and fall in the Monitoring and Logging topic.
Activity Log
Let's start with one that I believe to be the most important for every customer and that is the Activity Log. This feature provides you with a way to view the logging of every single Create, Update, List, or Delete action that is performed against a single subscription. These actions can be performed through the Portal, one of the Scripting mechanism (CLI or PowerShell) or through one of the many Azure SDKs (Software Development Kits) and they will show up as an entry within the Activity Log. By default, this service stores Activity Log entries for 90 days, but there are ways to export that information so that it can be archived for longer. So with that as a starting point, let's take a look at the Activity Log service:
[embed]https://vimeo.com/261478068/6aafbea3aa[/embed]
Diagnostics & Metrics
The next area of Azure Monitor to discuss is actually two items that I am going to combine together because they are both very tightly aligned with Logging and Alerting functionality and that is Diagnostics and Metrics. Every resource type within Azure has to ability to be monitored and logged, but each resource is going to provide different data and in some cases, those resources may need to be monitored in as close to real-time as possible and that is why we have two different capabilities.
The diagnostics function provided a way for certain types of logs to be stored where these logs contain pertinent pieces of information for a resource type within Azure. For example, the a Virtual Machine has both Boot and Guest OS diagnostics that can be turned on at creation time. These logs are placed into a Storage Account for retrieval later, but many other resource types within Azure also have diagnostic logs and how you retrieve them and then consume them is a big part of your monitoring and logging strategy.
On the flip side, the metric feature provides you with near real-time data about a particular resource. Going back to the Virtual Machine example, the diagnostic logs are going to be more heavily used for troubleshooting after something has already gone wrong. What if you wanted to be more pro-active and get a warning about something happening before it actually breaks? That is where metrics come into play, for some resource types within Azure, you have the ability to look at metric data, such as CPU percentage or bandwidth in or out. These are near real-time pieces of data that you can use and track to monitor the health of your resources and be more proactive.
Lets take a look at both of these features and how they can be leveraged from within the Azure Portal:
[embed]https://vimeo.com/287441670/f647a18ce7[/embed]
Network Watcher
The next area that I want to focus on isn't so much an area of monitoring and logging, but it does give you access to a lot of information pertaining to Azure Networking and some of it can easily be stored off like Diagnostic Logs. Network Watcher is more a set of tools to help you provide network troubleshooting capabilities like Packet Capture or Route validation and the like. Network Watcher provides a set of almost 10 tools that can provide information about how your network is working. Some of the tools are tied into Log Analytics which we will discuss in the next blog post, but most of them are available for free directly within the Azure Portal. A complete list of the tools can be found below:
- Network Topology
- Network Performance Monitor
- Connection Monitor
- IP Flow Verify
- Next Hop
- Security Group View
- VPN Diagnostics
- Packet Capture
- Connection Troubleshoot
Let's take a look at these different tools and how they can be used within the Portal:
[embed]https://vimeo.com/288061104/9d6b94c86a[/embed]
Alerting
The last area that we want to take a look at in this post is the Alerting capabilities within Azure Monitor. We have already taken a look at many of the logging features and all of these can be used to drive a particular Alert action, such as sending an email or triggering an Azure Function or Azure Automation Runbook, but how does this happen and how can you make sure that the right people are being notified.
Along with the Metrics features that we discussed above, the Alerting features were recently overhauled within Azure. So let's take a look at this new feature and function set within the Azure Portal:
[embed]https://vimeo.com/288066527/323e11d4e6[/embed]
Conclusion
I am sure that you noticed that I have not completely covered everything that was in the bulleted list at the top of the post and that is because there are just too many different features and functions that fall into the monitoring and logging capabilities to go into one post. In the next post, I will try and fill in the gaps and focus on the visualization of all this log data of which there are number of different features and functions within Azure that fall into that category as well.
One thing to keep in mind here, with all of these great capabilities, I can't really help you put together a generic strategy for which ones you use and which ones you don't, nor should you want me to. There are a lot of different questions that you need to ask yourself about how you can use all of these features that I have shown you, but at least now you have a starting point and will learn even more in the next post. For more information about each of the items that I have shown you, I have provided links to the main documentation pages for each one. See you next time.