CompTIA Cloud+ CV0-003 Exam: Basics of Logging, Monitoring, and Alerting

Objective: Given a scenario, configure logging, monitoring, and alerting to maintain operational status.

This guide explores three key topics: logging, monitoring, and alerting. In the “Logging” section you will learn about using collectors, categorizing logs by severity, performing log audits, and understanding different types of logs.
In the “Monitoring” section you will explore topics like baselines, thresholds, and tagging. You will also learn about log scrubbing and different types of monitoring, such as performance monitoring, resource utilization monitoring, and availability monitoring.
Lastly, in the “Alerting” section you will learn about common messaging methods and when you should disable alerts.

Topics:
- Logging
- Monitoring
- Alerting

1. SNMP version _____ provides key security features, such as authentication and encryption.
2. _____ is a feature in which metadata is applied to cloud resources.
3. True or false: You should disable alerts during software maintenance mode.

Answers:
1. 3
2. Tagging
3. True

Logging
In this section you will learn about logging tools and techniques that are commonly used in a cloud environment.

Collectors
A collector is a tool that is designed to gather log messages in a central location. The most common tools for collecting data on a network are the Simple Network Management Protocol and the Syslog protocol.

Simple Network Management Protocol (SNMP)
Simple Network Management Protocol allows devices to send log messages to a central point of collection. SNMP utilizes two primary components: SNMP agents and the SNMP manager.
The role of an SNMP agent is to collect log messages from devices and send these messages to the SNMP manager. The SNMP manager stores these logs in a database, provides the means to query these logs, and can be configured to send alerts to end users.
See the following section for a quick comparison of SNMP and Syslog.

Syslog
The Syslog protocol enables devices (and operating systems, like Linux) to send log messages to a Syslog server (that is, a Syslog collector). Queries can be performed on the Syslog server to discover problems on the system, but the server can also be configured to send alerts to end users.
Note that there isn’t any established rule that specifies what type of message would be associated with a severity level. This is left to the developers who create the alert messages on the device.

Syslog and SNMP perform similar functions, but there are a few key differences:
- If you are using version 3 of SNMP, there are some security features, including authentication. Syslog lacks security features.
- trap). Syslog relies on the device to send the log messages.
- SNMP also provides some control functionality in which actions can take place on the device as the result of a log message. Syslog is designed to aid in troubleshooting by allowing for the monitoring of log messages.

Analysis
As you can probably imagine, on a busy network the number of logs generated can be staggering. It is difficult for a human to be able to keep up with all of the log entries. As a result, most organizations use tools that analyze log data.
A tool that analyzes log data will typically use a baseline to determine normal operation of network devices. By reviewing log entries, the analytic tool can determine whether there is an issue on the network or with a specific network device. In the event that the tool recognizes a possible problem, an alert should be sent to a human to investigate further. In some cases the tool may also take direct action to attempt to fix the problem.

Dozens of tools are available for both SNMP and Syslog. The following is just a small example of some popular SNMP analytic tools:
- SolarWinds Network Performance Monitor
- LogicMonitor
- Site24x7
- Spiceworks Network Monitor

Severity Categorization
One component of Syslog is the capability of devices to define a severity of the log message. The following severities are supported:
- 0—Emergency
- 1—Alert
- 2—Critical
- 3—Error
- 4—Warning
- 5—Notice
- 6—Informational
- 7—Debug

Audits
A log audit (also called a log trail) is a collection of logs that provide a sequence of events. This information is very useful in determining which steps may have resulted in a problem or error.

Types
Logs are often broken into categories called types.
Examples of log types include
- Operating system logs
- Security logs
- Policy logs
- User-generated logs
- Data access logs
- Authentication logs
- Administrative action logs
- Application logs
Note that this isn’t a complete list of log types because each cloud environment may also have additional log types.
The following sections will provide more details about some of these log types.

Access/Authentication
Typically, whenever a cloud resource is accessed, like reading data from a bucket or connecting to a database, an access log is created. There are some exceptions because many cloud vendors allow logging to be turned on or off for resources, but the point is that an access log details when a resource is accessed in some way. This can also include when access is denied to a resource.
Access doesn’t always require authentication because some resources may be “world-accessible” or “world-readable.” When authentication does take place, a log entry should be created. This typically is not something that can be turned off and should definitely also include failed login attempts.

System
A system log is one that is generated by an action on an operating system. Often these logs are also found on the operating system, but in some cases they may be sent to a log collector.

Application
An application log is a log entry that is created by some sort of cloud-based application. It may be an application that your organization creates, or it might be an application that is provided by the cloud vendor.

Automation
Log automation is the process of automatically performing an action based on a specific log entry. For example, if someone attempts to log in as the administrator on one of your systems and this attempt fails, you may want to automate the process of sending a text alert to your IT staff.

Trending
In terms of logging, trending is determining what is taking place in your cloud environment by viewing logs over a period of time. For example, if you are concerned that you might run out of space in a cloud storage resource, you can look at the daily reports of space utilization over a period of time (such as the past 30 days) to determine whether you need to upgrade to a larger storage device.

Monitoring
The line between logging and monitoring can be blurred at times because often the tool that is used for logging also provides the function of monitoring. Consider logging to be the recording of what is happening, while monitoring provides an idea of how your environment is behaving and can provide you with alerts based on criteria that you define.
This section will focus on the different monitoring features that you should be aware of, including understanding the purpose of baselines, thresholds, and tagging.
A baseline is an established norm. Baselines are used to determine whether there may be a problem with a resource. Baselines are often established by analyzing data from logs over a period of time. Future activity is monitored to determine if a resource is functioning within the parameters of the baseline.

Thresholds
Thresholds are set to determine when an alert should be issued because an attribute of a resource has reached a maximum or minimum value. For example, performance of a web server running on a virtual machine might be important to ensure the web server is able to respond to client requests. This performance can be monitored by setting thresholds for key indicators, such as CPU utilization, free memory, and network performance. If the web server exceeds any of these thresholds, an alert would be issued, or an auto-scaling event (spanning another web server) could be implemented. Another example could be the threshold on cost of a cloud resource and an alert being sent to your email on reaching a predetermined limit.

Tagging
Tagging is a feature in which metadata is applied to cloud resources. This metadata typically includes key-value pairs. For example, a key could be “department,” and the value could be “sales.”
Tagging resources provides many advantages. For example, you could use tags to group together virtual machines to perform actions on all of the virtual machines. For instance, if you wanted to shut down a collection of virtual machines that were designed for a test of new database software, you could create a tag called “purpose” with a value of “database_test” for all of these resources. Then, to shut down all of these virtual machines, you could have the cloud management tool shut down all virtual machines with a value of “database_test” associated with the tag “purpose.”
Tagging can also be used when monitoring resources. You may want to create a threshold rule that applies to all the organization’s web servers. By applying the same tag value to all of these web servers, you can create a monitoring rule that applies to any resource with the tag value. This would also include any new resource that is created after you make the monitoring rule, assuming that it has the correct tag value.

Log Scrubbing
In some cases, sensitive information may be stored in logs. This is one of the reasons why your log data should be kept secure. But what if your log data is compromised? Wouldn’t it be better if sensitive information wasn’t stored in the logs?
You may not have a choice when it comes to what is initially stored in a log entry. Keep in mind that log information comes from a variety of sources, including applications and resources that may not provide you with the ability to customize what is logged. If you can modify what is sent to your logs and avoid placing sensitive data in the logs in the first place, this is the best course of action. When this approach isn’t possible, log scrubbing is the next best solution.
Log scrubbing is using a tool to search for and remove sensitive information. For example, suppose a log entry includes a customer’s Social Security number (hopefully, this would never happen!). You could use a log scrubbing tool to look for a number that matches the following pattern (N represents a single digit): NNN-NN-NNNN. If a value like this is matched, the log scrubbing tool can either delete it or replace it with a different value, such as “data removed.”

Performance Monitoring
Performance monitoring is designed to determine if your cloud resources are reacting at an acceptable level or standard that is provided by an SLA definition. Cloud resources are often distributed, which can have an impact on performance. Additional cloud attributes may also have an impact on performance, including network security, hardware resources, and the configuration options of applications.

Application
Performance monitoring is often associated with a collection of tools called Application Performance Monitoring (APM). These APM tools can be used to determine how well the cloud resource is responding and help you determine where a bottleneck may be causing poor performance.

Infrastructure Components
Infrastructure components is a broad categorization of resources that can include the network itself, hardware components (CPU, memory, and so on), facilities (power, for example), and software. These are typically the sorts of components that can have an impact on performance and are the primary resources that are monitored with performance monitoring tools.

Resource Utilization
In the context of resource utilization, consider components like CPUs, memory, network bandwidth, and disk space to be the resources. The more resources your cloud infrastructure uses, the higher the costs of your cloud environment.
Resource utilization monitoring is designed to determine what parts of your cloud infrastructure are using these resources. The goal is to help you make better decisions of how to adjust your cloud environment to best optimize the resources that are available.

Availability
As you can probably imagine, many of the components of your cloud infrastructure need to be available almost 100 percent of the time. While 100 percent would be most ideal, there is almost always the chance of short periods of downtime. The goal is to keep this downtime as short as possible.
To determine whether a cloud resource is available, you can use monitoring tools to actively or passively check for this availability. The results should be compared to the uptime minimums that are defined in the cloud vendor’s SLA (see “Service-Level Agreement (SLA)” in “Solution Design in Support of the Business Requirements”).

SLA-Defined Uptime Requirements
See the “Service-Level Agreement (SLA)” section in this guide.

Verification of Continuous Monitoring Activities
Imagine a scenario in which you have opted to use a company to monitor the alarm system that you have installed in your house. You put your trust in this organization monitoring the status of your alarm to keep you and your house safe.
Now imagine that your alarm goes off one night and…nothing happens. No response from the alarm monitoring company, no police dispatched to your house, just no response at all. Clearly, this situation is something that you want to avoid, so you take care to pick a reputable company to monitor your alarm and ensure that it is routinely performing checks to verify that the alarm monitoring system is functioning correctly.
This same care needs to be established with the tools that you deploy to monitor your cloud infrastructure. This verification could include manual audit checks, alarms raised when a cloud resource is no longer providing data, or tools that run checks on your monitoring software.

Service Management Tool Integration
Service management, also called information technology service management (ITSM), is the means by which an organization provides IT services to its customers (which also include employees). When an organization uses the cloud to provide services, the monitoring solutions that are deployed need to be integrated with the service management tools to provide timely response times.
For example, suppose that an organization provides a service to its customers that is based on a cloud-based database. Clearly, this service becomes unusable if the database is no longer available. The tool that is used to monitor the database availability must be configured to “talk to” the ITSM software to provide IT support with accurate, up-to-the-minute information on the availability of the service.

Alerting
Alerting is one of the actions that monitoring tools can take when a threshold is reached or an error is encountered in your cloud environment. This section will explore some of the key components of alerting.

Common Messaging Methods
Where you send an alert can be as important as determining which alerts you want to send. Following are some of the common alert messaging methods that you typically find in cloud environments:
- SMS (that is, a text message to your mobile device)
- A messaging service, like AWS Simple Notification Service (SNS), PagerDuty, or Datadog
- A server via an API call
- A storage container, like a bucket
- A messaging tool, like Microsoft Teams or Slack

Enable/Disable Alerts
In some cases you should not have alerts enabled. For example, if a problem generates a massive number of alerts, you may want to disable the alert system until that problem is resolved.
Additionally, when a cloud service or resource is in maintenance mode (such as updating software or resolving bugs), disabling alerts prevents false warnings. It is very likely that software that is in maintenance mode will trigger alerts as the software is being worked on.

Maintenance Mode
See the preceding “Enable/Disable Alerts” section.

Appropriate Responses
Some alerts might just require additional research and monitoring, whereas others may require immediate action. Developing a solid response plan is just as important as determining which alerts to generate and where to send the alerts.
Included in developing a response plan is developing a policy for categorizing and communicating alerts. See the next section for more details.

Policies for Categorizing and Communicating Alerts
Not all alerts carry the same importance. For example, an alert indicating that a key database is no longer available should have a higher importance than an alert indicating that the same database may need more storage space in the near future.

You should develop a policy for categorizing alerts. This policy can be based on the level of importance of the alert or other criteria, like the following:
- The type of service or resource that is impacted by the alert (operating system, application, database, storage, network performance, and so on)
- Who should be notified based on the alert
- If the alert affects key customers or general overall service
- If the alert is related to a key service that impacts other resources or services
Once you have placed alerts into categories, your policy should be expanded to include how to communicate the alerts after they have been received. This includes determining when an alert needs to be escalated and who should be notified when this situation occurs.

Quiz:
1. SNMP can gather messages either by polling devices or by having devices send messages via a(n) _____. A.Trap B.Alert C.Notification D.Push
2. Which of the following is the highest severity level of Syslog? A.Critical B.Emergency C.Alert D.Error
3. Log scrubbing is designed to remove what type of information? A.Sensitive B.Server C.Redundant D.Outdated
4. Which of the following is not normally a destination of an alert message? A.Email B.SMS C.Phone call D.Storage container

Answers:
1. Trap
2. Emergency
3. Sensitive
4. Phone call