CompTIA Linux+ Certification: Optimizing Performance

Key Topics:
Looking at Processes
Monitoring Processes in Real Time
Managing Processes

Objective 1.4: Given a scenario, configure and use the appropriate processes and services.
This guide discusses how Linux handles applications running on the system. Linux must keep track of lots of different programs, all running at the same time. Your goal as the Linux administrator is to make sure everything runs smoothly. This guide shows just how Linux keeps track of all the active programs and how you can peek at that information. You'll also see how to use command-line tools to manage the programs running on your Linux system.

Looking at Processes
At any given time a large number of active programs are running on the Linux system. Linux calls each running program a process. A process can run in the foreground, displaying output on a console display or graphical desktop window, or it can run in the background, working on data behind the scenes. The Linux system assigns each process a process ID (PID) and manages how the process uses memory and CPU time based on that PID.
When a Linux system first boots, it starts a special process called the init process. The init process is the core of the Linux system; it runs scripts that start all the other processes running on the system, including the processes that start the text consoles and graphical windows you use to log in (read “Maintaining System Startup and Services”).
You can watch just which processes are currently running on your Linux system by using the ps command.

The default output of the ps command looks like this:
$ ps
PID TTY TIME CMD
2797 pts/0 00:00:00 bash
2884 pts/0 00:00:00 ps

By default, the ps command only shows the processes that are running in the current user shell. In this example, we only had the command prompt shell running (Bash) and, of course, the ps command itself.
The basic output of the ps command shows the PID assigned to each process, the terminal (TTY) that they were started from, and the CPU time that the process has used.
The tricky feature of the ps command (and what makes it so complicated) is that at one time there were two versions of it in Linux. Each version had its own set of command-line parameters controlling the information it displayed. That made switching between systems somewhat complicated.
Recently, the GNU developers decided to merge the two versions into a single ps program, and of course, they added some additional parameters of their own.

The current ps program used in Linux supports three different styles of command-line parameters:
Unix-style parameters, which are preceded by a dash
BSD-style parameters, which are not preceded by a dash
GNU long parameters, which are preceded by a double dash

This makes for lots of possible parameters and options to use with the ps command. You can consult the ps man page to see all the possible parameters that are available. Most Linux administrators have their own set of commonly used parameters that they remember for extracting pertinent information.

For example, if you need to see every process running on the system, use the Unix-style -ef parameter combination, like this:
$ ps -ef

UID PID PPID C STIME TTY TIME CMD
root 1 0 0 09:02 ? 00:00:02 /sbin/init
root 2 0 0 09:02 ? 00:00:00 [kthreadd]
root 3 2 0 09:02 ? 00:00:00 [ksoftirqd/0]
root 4 2 0 09:02 ? 00:00:00 [kworker/0:0]
root 5 2 0 09:02 ? 00:00:00 [kworker/0:0H]
root 6 2 0 09:02 ? 00:00:00 [kworker/u2:0]
root 7 2 0 09:02 ? 00:00:02 [rcu_sched]
root 8 2 0 09:02 ? 00:00:01 [rcuos/0]
root 9 2 0 09:02 ? 00:00:00 [rcu_bh]
root 10 2 0 09:02 ? 00:00:00 [rcuob/0

This format provides some useful information about the processes running:
UID: The user responsible for running the process
PID: The process ID of the process
PPID: The process ID of the parent process (if the process was started by another process)
C: The processor utilization over the lifetime of the process
STIME: The system time when the process was started
TTY: The terminal device from which the process was started
TIME: The cumulative CPU time required to run the process
CMD: The name of the program that was started in the process

Also notice in the -ef output that some process command names are shown in brackets. That indicates processes that are currently swapped out from physical memory into virtual memory on the hard drive. Processes that are swapped into virtual memory are called sleeping. Often the Linux kernel places a process into sleep mode while the process is waiting for an event.
When the event triggers, the kernel sends the process a signal. If the process is in interruptible sleep mode, it will receive the signal immediately and wake up. If the process is in uninterruptible sleep mode, it only wakes up based on an external event, such as hardware becoming available. It will save any other signals sent while it was sleeping and act on them once it wakes up.

Note: zombie. It's stuck in a limbo state between running and terminating until the parent process acknowledges the termination signal.

Monitoring Processes in Real Time
The ps command is a great way to get a snapshot of the processes running on the system, but sometimes you need to see more information to get an idea of just what's going on in your Linux system. If you're trying to find trends about processes that are frequently swapped in and out of memory, it's hard to do that with the ps command.
Instead, the top command can solve this problem. The top command displays process information similar to the ps command, but it does it in real-time mode.

Figure below shows a snapshot of the top command in action.

Figure: The output of the top command

The first section of the top output shows general system information. The first line shows the current time, how long the system has been up, the number of users logged in, and the load average on the system.
The load average appears as three numbers: the 1-minute, 5-minute, and 15-minute load averages. The higher the values, the more load the system is experiencing. It's not uncommon for the 1-minute load value to be high for short bursts of activity. If the 15-minute load value is high, your system may be in trouble.
The second line shows general process information (called tasks in top): how many processes are running, sleeping, stopped, or in a zombie state.
The next line shows general CPU information. The top display breaks down the CPU utilization into several categories depending on the owner of the process (user versus system processes) and the state of the processes (running, idle, or waiting).


TABLE:  The top CPU categories
 

Following that, there are two lines that detail the status of the system memory. The first line shows the status of the physical memory in the system, how much total memory there is, how much is currently being used, and how much is free. The second memory line shows the status of the swap memory area in the system (if any is installed), with the same information.

Finally, the next section shows a detailed list of the currently running processes, with some information columns that should look familiar from the ps command output:
PID: The process ID of the process
USER: The username of the owner of the process
PR: The priority of the process
NI: The nice value of the process
VIRT: The total amount of virtual memory used by the process
RES: The amount of physical memory the process is using
SHR: The amount of memory the process is sharing with other processes
S: The process status (D = interruptible sleep, R = running, S = sleeping, T = traced or stopped, and Z = zombie)
%CPU: The share of CPU time that the process is using
%MEM: The share of available physical memory the process is using
TIME+: The total CPU time the process has used since starting
COMMAND: The command-line name of the process (program started)

By default, when you start top it sorts the processes based on the %CPU value. You can change the sort order by using one of several interactive commands. Each interactive command is a single character you can press while top is running and changes the behavior of the program. These commands are shown below.

TABLE:  The top interactive commands

You have lots of control over the output of the top command. Use the F or O command to toggle which field the sort order is based on. You can also use the r interactive command to reverse the current sorting. Using this tool, you can often find offending processes that have taken over your system. A relatively new utility, the htop program is an improved version of the top utility, available in most Linux distributions as an additional software package that you can install.

Figure below shows an example of the htop output display.

Figure: The output from the htop command tree view

The htop output uses color-coding to make things easier to read, along with use of the mouse to select items. Instead of cryptic letters for actions, it uses function keys to easily perform display functions, such as sorting or filtering the output, searching the output, or arranging the output in a tree to easily view parent and child processes. Perhaps one of its more popular features that makes it improved over top is the ability to customize the output display so that you only see the process information columns you're interested in monitoring.

Managing Processes
One of the jobs of a Linux system administrator is to be on the watch for runaway processes that can take down the Linux system. You've already seen how to use the ps and top commands to monitor how processes are doing on the system; the next step is to see how to stop a runaway process.

Setting Priorities
By default, all processes running on the Linux system are created equal; that is, they all have the same priority to obtain CPU time and memory resources. However, you may run some applications that either don't need to have the same level of priority or may need a higher level of priority.
The nice and renice commands allow you to set and change the priority level assigned to an application process. The nice command allows you to start an application with a nondefault priority setting.

The format looks like this:
nice -n value command

Note: nice or renice commands.

The value parameter is a numeric value from –20 to 19. The lower the number, the higher priority the process receives. The default priority is 0. The command parameter indicates the program you want to start at the specified priority:
$ nice -n 10 ./myscript.sh
 

To change the priority of a process that's already running, use the renice command:
renice priority [-p pids] [-u users] [-g groups]
The renice command allows you to change the priority of multiple running processes based on a list of PID values, all of the processes started by one or more users, or all of the processes started by one or more groups.
$ renice 15 -p 3178
 

Warning: - Only the root user account can set a priority value less than 0 or decrease the priority value (increase the priority) of a running process.

Stopping Processes
Sometimes a process gets hung up and just needs a gentle nudge to either get going again or stop. Other times, a process runs away with the CPU and refuses to give it up. In both cases, you need a command that will allow you to control a process. To do that, Linux follows the Unix method of interprocess communication.
In Linux, processes communicate with each other using process signals. A process signal is a predefined message that processes recognize and may choose to ignore or act on. The developers program how a process handles signals. Most well-written applications have the ability to receive and act on the standard Unix process signals. These signals are shown below.

TABLE:  Linux process signals

While a process can send a signal to another process, there are two commands available in Linux that allow you to send process signals to running processes.

The kill Command
The kill command allows you to send signals to processes based on their process ID (PID). By default, the kill command sends a SIGTERM signal to all the PIDs listed on the command line. Unfortunately, you can only use the process PID instead of its command name, making the kill command difficult to use sometimes.
To send a process signal, you must either be the owner of the process or be logged in as the root user.
$ kill 3940
-bash: kill: (3940) - Operation not permitted
$

The SIGTERM signal only asks the process to kindly stop running. Unfortunately, if you have a runaway process, most likely it will ignore the request. When you need to get forceful, the -s parameter allows you to specify other signals (either using their name or using their signal number).
The generally accepted procedure is to first try the TERM signal. If the process ignores that, try the SIGINT or SIGHUP signal. If the program recognizes these signals, it will try to gracefully stop doing what it was doing before shutting down. The most forceful signal is the SIGKILL signal. When a process receives this signal, it immediately stops running. Use this as a last resort, as it can lead to corrupted files.

One of the scary things about the kill command is that there's no output from it:
$ sudo kill -s SIGHUP 3940
To see if the command was effective, you'll have to perform another ps or top command to see if the offending process stopped.
lsof command first to see a list of the open files and their processes.

The pkill Command
The pkill command is a powerful way to stop processes by using their names rather than the PID numbers. The pkill command allows you to use wildcard characters as well, making it a very useful tool when you've got a system that's gone awry:
$ sudo pkill http*
This example will kill all of the processes that start with http, such as the httpd services for the Apache web server.
 

pkill command. It's usually a good idea to check the search term against the currently running processes to make sure you don't accidentally kill any other processes that match the search. The pgrep command allows you to display all processes that match the search term. You can also use the pidof command to view the PID of a specified program to ensure you have the correct PID and application.
The following exercise demonstrates how to monitor the running processes on your Linux system and how to remove a process you no longer want running.

EXERCISE: Managing a Running Process

1. Log into your Linux graphical desktop and open two new command prompt windows. If you're using virtual terminals, open two separate virtual terminal sessions.
2. In the first command prompt, enter a command to run the sleep program for 1000 seconds by typing sleep 1000.
3. In the second command prompt window or virtual terminal session, look for the PID of the sleep program by typing pgrep sleep.
4. Once you know the PID, use the kill command to stop it prematurely by typing sudo kill -SIGHUP pid, where pid is the PID of the sleep program you found in step 3.
5. Observe the command prompt in the first window. It should return, indicating that the sleep program is no longer running.
6. Check the running processes to ensure that the command is no longer running by typing pgrep sleep.

Managing applications running on Linux systems is a crucial job for administrators. The Linux system allocates CPU time and memory for each process, and knowing how each process consumes those resources is helpful. You can view the running applications and the resources they consume by using the ps command. There are many different ways to view process information using the ps command, allowing you to customize the display exactly how you like.
For real-time monitoring of applications, use the top command. With the top command, you can view a real-time display of applications, their system state, and the resources they consume. The top command allows you to sort the display based on many different features, such as CPU usage or memory usage.
For managing applications as they run on the system, you can use the nice and renice commands. The nice command allows you to start an application at a different priority level than the applications that are already running. This allows users to run applications in the background at a lower priority or allows the system administrator to start applications with a higher priority. With the renice command, you can change the priority of an application that's already running.
If an application causes problems and needs to be stopped, you can use the standard Linux kill command, but you need to know the process ID assigned to the application by the system. The pkill command is customized for stopping applications by their name instead of process ID. You can also use wildcard characters to stop multiple applications, but doing so can be dangerous. To test which applications would be stopped, use the pgrep command to search for running applications using the wildcard search.

Important exam questions

1. Explain how to view the status of applications running on the Linux system.
The ps and top commands display the status of applications running on the system. Both commands display the status of each application, whether it's running, sleeping, or waiting for resources.

2. Explain how you would find what applications are using the most resources on your system.
The top command allows you to monitor the applications running on the Linux system in real time. It displays the amount of CPU and memory each application is consuming and allows you to sort the display on any of the displayed data fields. This allows you to easily monitor which applications are using the most CPU time or the most memory at any given moment.

3. Describe how you can stop an application that's causing problems on the Linux system.
Linux applications are programmed to respond to Linux signals. The kill command allows you to send Linux signals to running applications. If you send a KILL signal to an application, it will stop running on the system. Alternatively, you can send an INT signal to interrupt the application, allowing you to close it down gracefully if the application responds. The pkill application allows you to send Linux signals to applications based on their process name and lets you specify the process name using wildcard characters. This combination can come in handy if you need to stop multiple applications spawned from a single parent application.