CompTIA Linux+ Certification: Investigating User Issues

Key Topics:
Troubleshooting Access
Examining File Obstacles
Exploring Environment and Shell Issues

Objective 4.4: Given a scenario, analyze and troubleshoot user access and file permissions
An old troubleshooting technique is to break a problem in half: if the cause is not in the first half, it's in the second half. Break the second part in half and continue to analyze. In this guide, we're splitting user issues into system access and file difficulties. We further divide these problem categories while stepping you through various items to consider as you move toward a solution.

Troubleshooting Access
If a user is having trouble accessing their desired applications, a mixed bag of authentication issues can be the cause. We'll take a look at local and remote access as well as authentication scenarios.
Local access refers to those users who are using a directly connected interface to the server. These are typically server administrators but may be application users as well.

Begin troubleshooting by gathering some basic information:
Is this a newly created user account?
What is the username being entered for the account?
Has the user ever logged into the account?
Is the user attempting to log in via the GUI or a text-based (virtual) terminal (for example, tty2)?

Checking a Newly Created User Account
If the account is a newly created account, confirm that it was properly built. New system administrators often create user accounts with the useradd command (Read “Administering Users and Groups”) but forget to add its password with the passwd utility. Use either the grep or the getent command to check the /etc/passwd and /etc/shadow file records. An example is shown in List 22.1 for a new user account, JKirk, on an Ubuntu Desktop distribution.

List: Viewing a user account record with the getent command
$ sudo getent passwd JKirk
Notice that in the password field for the JKirk shadow record, there is an exclamation mark (!). This indicates a password was not created for the account.
 

Tip: - Make sure your system users know that usernames are case sensitive on Linux. Other operating systems, such as Windows, have usernames that are case insensitive, and this can cause confusion.

Checking Account Accesses
Look at the last time the account was successfully accessed. The lastlog utility searches through the /var/log/lastlog file for users who have logged into the system, but it only maintains the most recent login. The last command searches the /var/log/wtmp file for users who have logged in/out and keeps records for the most recent logins and beyond. This file is typically rotated with earlier versions and given a numeric extension. Thus, the next oldest version of wtmp is wtmp.1.

The last command's -f option will help you search through the various files, as shown below.

List: Viewing successful logins with the last command
$ sudo last -f /var/log/wtmp -f /var/log/wtmp.* | grep JPicard
JPicard tty3 Wed Jan 4 13:14 - 13:14 (00:00)
$ sudo lastlog -u JPicard
Username Port From Latest
JPicard tty3 Wed Jan 4 13:14:11 -0500 2023

Notice the last command shows that the JPicard account was logged into on January 4 at the tty3 terminal, and this is confirmed via the lastlog command.
You can also employ the lastb command. This command allows you to search for unsuccessful login attempts, as shown below. Notice that the JPicard account had two failed login attempts at the tty4 terminal.

List: Viewing unsuccessful login attempts with the lastb command
$ sudo lastb -f /var/log/btmp -f /var/log/btmp.* | grep JPicard
JPicard tty4 Wed Jan 4 13:29 - 13:29 (00:00)
JPicard tty4 Wed Jan 4 13:29 - 13:29 (00:00)

Checking Privilege Elevation Issues
Most Linux distributions allow use of the sudo command to let users perform administrative tasks as the root user account, which is called privilege elevation. If there's a user account that's not allowed to use the sudo command, you need to check a few things.
First, take a look at the /etc/sudoers file to see what users and groups are allowed to elevate their privileges. If the user account isn't specified directly in the /etc/suoders file, there may be some groups that are specified.

Those would have entries that look like this:
# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL
# Allow members of group sudo to execute any command
%sudo ALL=(ALL:ALL) ALL

In this configuration from an Ubuntu system, users in the admin group are allowed to use the sudo command for root privilege elevation, and members of the sudo group can gain privileges to execute any command as any user account.

The next step is to see what groups the user account belongs to. The easiest way to do that is by using the id command:
$ id rich
uid=1000(rich) gid=1000(rich) groups=1000(rich),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),
120(lpadmin),131(lxd),132(sambashare)

The output from the id command shows all of the groups the user account belongs to. In this example, the rich user account does belong to the sudo group, so it should have access to use the sudo command for privilege elevation as any user account on the system.

Checking GUI Issues
If you find that the account successfully logged into the system in the past and has no recent failed attempts, find out the user's local access method. If using the GUI, have the user attempt to log into a text-based terminal, such as the tty2 terminal. If the user cannot successfully log into a terminal, then something else is amiss. 

If the user can successfully log into a text-based terminal but not the GUI, you've narrowed down the problem. Determine what services are running. For older SysVinit systems, the runlevel command will show whether the graphical environment is current (see “Maintaining System Startup and Services”). For systemd systems, use the command shown in the snipped examplebelow.

List: Viewing the current systemd target using the systemctl command
$ sudo systemctl status graphical.target
graphical.target - Graphical Interface
Active: active since Wed [...]

The graphical.target is active, which indicates GUI services are available. Thus, providing GUI services is not the problem. Begin investigating other potential GUI issues, starting with the display manager ( Read “Comparing GUIs”).

Note: multi-user.target is active. Read here  for SysVinit systems.

Checking Terminal Issues
If the user typically logs into a text-based terminal but can't, have them log into a different terminal, either at a different virtual terminal or a graphical desktop. If the login is successful, then look at the original terminal's device file to determine if it is corrupted by using the ls -l command.

An example is shown snippedbelow.

List: Viewing terminal device files with the ls -l command
$ ls -l /dev/tty?
crw--w---- 1 root tty 4, 0 Jan 4 09:38 /dev/tty0
crw--w---- 1 gdm tty 4, 1 Jan 4 09:38 /dev/tty1
crw--w---- 1 root tty 4, 9 Jan 4 09:38 /dev/tty9

Notice the c at the beginning of each terminal's device file record. This indicates the device file is a character file. If you see a dash (-) instead, the file is corrupted. Rebuild it using super user privileges and the mknod command.

Note: Read “Analyzing System Properties and Remediation”  - in the “Surviving a Lost Root Password” section.
If the user attempts to log into a different text-based terminal and can't, check to see if getty services are running. These services provide the login prompts for the text-based terminals. An example of checking for getty services on a systemd system is shown snipped below.

List: Checking for getty services with the systemctl command
$ sudo systemctl status getty.target
getty.target - Login Prompts
Active: active since Wed 2023-01-04 09:38:13 EST; 6h ago

The getty.target is active, so getty services are available. If a user still cannot log into the system, you'll need to explore additional issues.

Note: /etc/security/access.conf file. This file is scanned when a user attempts to log into the system. Its configuration accepts or blocks users/groups from accessing the system. It can also prohibit certain logins to the text-based terminals, as well as logins originating over the network.

Checking Additional Local Issues
Determine if the account is locked. You can employ the passwd -S or the getent command to check this, as shown snipped below.

List: Checking if an account is locked with the passwd and getent commands
$ sudo passwd -S KJaneway
KJaneway L 01/02/2019 0 99999 7 -1
$ sudo getent shadow KJaneway
KJaneway:!$6$[...]0:17898:0:99999:7:::

The L after the user KJaneway account's name indicates the account is locked. However, that code is also shown for accounts that have no password set. Thus, the getent command is also employed. The exclamation point (!) at the front of the account password's field verifies that the account is indeed locked. To unlock the account, if desired, use super user privileges and the usermod -U or the passwd -u command.
The account may have expired. Account expiration dates are typically set up for temporary account users, such as contractors or interns. You can view this information using the chage command, as shown snipped below.

List: Checking if an account is expired with the chage command
$ date
Wed Jan 4 16:17:48 EST 2023
$ sudo chage -l JArcher
Account expires : Jan 01, 2023

Notice that this account's expiration date has passed. Therefore, the JArcher account is now expired and the user cannot log into it. If this was a mistake or you need to modify it, use super user privileges and the chage -E command to set a new expiration date for the account.
Confirm that the user is using the correct password, and check if the account's password is expired. Employ the chage -l command to view this as well.
For remote access problems, first check that the system is accessible over the network (network troubleshooting was covered here). If the system is accessible, determine how the user is attempting to access the system.
If the user is employing OpenSSH, first confirm that the OpenSSH server is running on the system and that the firewall is properly configured to allow access (firewalls were covered in “Overseeing Linux Firewalls”). Next review the sshd_config configuration file. The AllowUsers and AllowGroups directives restrict access. Ensure that these are correctly set. In addition, verify that there are no specific override settings for this particular user at the file's bottom. Be sure to review any configuration files on the client side as well, such as ~/.ssh/config and /etc/ssh/ssh_config (OpenSSH was covered in “Looking at Access and Authentication Methods”).
Tip: -vvv option onto their ssh command. This provides a great deal of verbose information, which may help in the troubleshooting process.
Determine whether authentication through OpenSSH is via a username and password or via a token. If it is a username/password, check that the sshd_config directive PasswordAuthentication is set to yes. If all is well with the configuration file, troubleshooting topics in the next section may help.
If the OpenSSH authentication is token based, ensure that the private key was properly copied over to the server from the remote system. Also, confirm that the public key is stored in the ~/.ssh/authorized_keys file on the client side.

Note: ForwardX11 directive is set and that the user is employing the -X option with the ssh command. See here for additional details.
If your users are employing a remote desktop application, such as VNC, xrdp, NX, or SPICE, review their configurations. More information can be found on these topics here.


Authentication
Layered authentication software could be at the problem's heart. One place to check is PAM (covered here). Look through the PAM configuration files, such as /etc/pam.d/sshd, to ensure that directives are properly set. Also employ the pam_tally2 or faillock utility to check if the user's account is locked due to too many failed login attempts.
Does your system employ other authentication products, such as LDAP (covered here) or Kerberos (covered in “Embracing Best Security Practices”)? You'll want to check their configuration files. Also take a look through their log files to ensure that a policy violation has not locked out the user's account.
Don't forget to check your Linux kernel security module, such as SELinux or AppArmor (covered in “Applying Ownership and Permissions”). While the purpose of these modules is to protect the system from attackers, sometimes policy violations can lock out legitimate users. For AppArmor, policy violations are stored in either audit.log (produced by auditd) or messages.log, depending on its configuration. If using the auditd service, you can search for AppArmor policy violations in the audit.log file using the ausearch command.

For SELinux, check the audit log file using the sealert command, as shown in List 22.9. Notice that no SELinux policy violations were logged.

List: Checking SELinux policy violations with the sealert command
# sealert -a /var/log/audit/audit.log
100% done
found 0 alerts in /var/log/audit/audit.log

Tip: id -Z command to view a user's SELinux context.

For authentication issues, peruse through these various log files as well:
/var/log/auth.log (Debian-based distros)
/var/log/secure (Red Hat–based distros)

These log files typically contain information such as authentication failures and can provide you with a great deal of information. Use grep to search for a particular username, if desired.

Examining File Obstacles
Various problems may ensue when managing files. Understanding the typical problems will help in your file troubleshooting efforts.

File Permissions
You're editing a configuration file using the vim editor, but when you try to save the file, you get a “Can't open file for writing” error message. That's frustrating. Problems like this are directly related to file permissions.

When encountering these issues, first use the ls -l command to view a file's permission settings and ownerships. Note the file's owner and group. Then determine the permissions of the owner, group, and everyone else (world or other).

An example is shown below.

List: Determining file permissions with the ls -l command
$ ls -l HelloWorld.sh
-rwxr-xr--. 1 root wheel 72 Jan 4 09:29 HelloWorld.sh

In this case, the file is owned by root, who has permission to read, write to, and execute the file. The file's group is wheel. Those members can read and execute the file. Finally, everyone else who is not the file's owner or does not belong to the file's group can only read the file.
If a problem occurs trying to access this file, determine the user's username and group memberships using the id command. Match their identity against the file's permissions to find the problem. 

List: Troubleshooting file access with the id command
$ ./HelloWorld.sh
-bash: ./HelloWorld.sh: Permission denied
$ id Christine
uid=1001(Christine) gid=1001(Christine) groups=1001(Christine)

Christine cannot execute this file because she is not the file's owner, nor is she in the file's group (see the list) The third permission set applies to her and only allows her to read the file. To let her run the file, add her to the wheel group, set her as the file's owner, or modify the file's world permissions.

Directory Permissions
While the directory permission settings look very similar to file permissions, their effect is different.

Table below shows permission effects on actions (covered here) associated with the entity to whom the permission applies (owner, group, or other).

TABLE:  Directory permission effects
 

A few examples will help you to understand these permissions. For example, the user Christine can list the /etc directory's contents, due to the directory's read permission for other (world), as shown below.

List: Understanding directory permission effects of the read privilege
$ ls -ld /etc
drwxr-xr-x. 143 root root 12288 Jan 4 09:22 /etc
$ ls /etc
abrt gshadow- profile
adjtime gss profile.d

The next example, the list shows that the parent directory (/home) does allow a user to use cd to change their present working directory to it. However, a subdirectory (Samantha) blocks it by not granting the execute (x) privilege.

List: Understanding directory permission effects of the execute privilege
$ ls -ld /home/
drwxr-xr-x. 5 root root 52 Nov 8 14:31 /home/
$ cd /home
/home
$ ls -ld /home/Samantha
drwx------. 5 Samantha Samantha 128 Nov 8 14:38 /home/Samantha
$ cd /home/Samantha
-bash: cd: /home/Samantha: Permission denied

Note: not be able to use cd to get to that subdirectory. All directories in that path must have the execute permission set on them for that user.
If you have a directory shared among users and they are able to delete each other's files but that is not desired, employ the sticky bit on the directory. This permission (covered here) will solve the problem.

Working with Advanced Permissions
Besides the standard Linux file and directory permissions, you may run into a system that utilizes one of the more advanced Linux permission features.
Access control lists (ACLs) allow you to specify permissions for multiple users or groups besides the standard owner, group, and others set. If your Linux system utilizes ACLs, always check the getfacl utility for additional ACL permissions applied to a file or directory:
$ getfacl test.txt
# file: test.txt
group::r--

In this example the owner has assigned sales group read permissions to the file using ACLs. To change ACL settings, use the setfacl utility.
Besides ACL permissions, Linux systems may also employ context-based permissions using either SELinux (most common in Red Hat–based Linux distributions) or AppArmor (most common in Debian-based Linux distributions).

To determine if SELinux is active on your Linux system, use the sestatus command:
Loaded policy name: targeted
Max kernel policy version: 331
If SELinux is enabled, use the -Z command-line option for the ls command to view additional context permissions for files and directories:

You can change the security context for a file by using the chcon command:
SELinux controls access to objects using policies. The getsebool command allows you to list the policies and determine if they're active or disabled, and use the setsebool command to activate or deactivate them.

To determine if AppArmor is active on your system, use the aa-status utility:
$ sudo aa-status

AppArmor uses profiles to set context permissions. While this output has been truncated, in the actual aa-status output you'll see lots of information about the profiles loaded in the enforce, complain, and disable modes. Use the aa-enforce, aa-complain, or aa-disable command to change profile settings to accommodate your permission requirements.

File Creation
A user goes to create a file and permission is denied. First check the directory permissions. If all is well there, consider the following items:
Are quotas enforced on this partition?- The user may have hit a quota limit. Check the partition's /etc/fstab record for either usrquota or grpquota to see if quotas are enabled on this filesystem (covered here) or use the repquota utility. If quotas are enabled, check the user's (or group's) quota usage using the quota command. If the user has exceeded quota limits and the grace period has passed, turn off quotas with quotaoff, extend the user's quotas, or have the user delete unneeded files.
Has the disk run out of space?- The df command (covered here ) can help you quickly determine if the partition on which the user is attempting to create a file has run out of space. If you are out of space, take the appropriate action (also covered here)
Has the partition run out of inodes?- Inode exhaustion is an unusual situation, but it does happen. An inode number (covered in “Managing Files, Directories, and Text”) is a file's associated index number. If a filesystem runs out of inodes, no additional files can be created on it. Check if this is the problem by employing the df -i command, which shows each filesystem's inode use.
If inodes are exhausted, you cannot extend them after filesystem creation. Instead, look for directories that have a large amount of small unnecessary files, such as a temporary directory, and remove what you can. A good practice is to put applications that create many small files on their own large partitions formatted to provide higher amounts of inode numbers. At filesystem creation, increase inode counts above their defaults by using utility options like the mke2fs -i command.
What is the user's umask setting?- The umask setting (covered here) subtracts permissions from the default file and directory permission settings. Thus, when a user is creating files and/or directories, the permissions set on them are affected by this setting. Have the user enter umask at the command line (or check the user's environment file) to determine its setting. If it is set too high (for example, removing needed write permissions for created subdirectories), you have found the cause of the problem.

Tip: lsattr filename command. If you see an i among the dashes preceding the filename, the file has the immutable bit set. This bit prevents even users with super user privileges from deleting the file. To remove the file's immutable bit, use super user privileges and issue the chattr -i filename command.
Just as with access issues, check your Linux kernel security module (SELinux or AppArmor) in situations where a user cannot create or delete files. Peruse the appropriate policy violation log files (/var/log/messages or /var/log/audit/audit.log) using the appropriate tools.

Note: auditd, the SELinux policy violation messages are stored in the /var/log/messages file.
After you have found the policy violation concerning the file or directory in question, determine if the file/directory was mislabeled, the policy is incorrect, or possibly the wrong security context was used. Take appropriate actions to remedy the situation.

Exploring Environment and Shell Issues
When dealing with user problems, a potential issue is the user account's default shell. Check it using the getent command, as shown on an Ubuntu distribution below.

List: Determining an account's default shell with the getent command
$ getent passwd BSisko
BSisko:x:1007:1007::/home/BSisko:/bin/sh
$ readlink -f /bin/sh
/bin/dash

Notice that instead of the /bin/bash shell as the default shell, the /bin/sh shell is used. On this system, the /bin/sh file is linked to the /bin/dash shell. If this is not desired, you can change the user's default shell via the usermod command (read this guide)

Note: /sbin/false, /sbin/nologin, or something similar. These shells are typically for daemons and prevent the daemon from logging into the system.
Incorrect or improperly set environment variables can cause various user difficulties. Review a user's environment files (covered here), such as the ~/.profile file. While you're at it, check the system's global environment variable settings using the set, env, or printenv command.
Ensure that environment variables are exported so that they are available in subshells (a subshell may occur when a shell script is executed.) An example of this problem is shown below.

List: Demonstrating what happens when variables are not exported
$ EDITOR='nano'
$ echo $EDITOR
nano
$ bash
$ exit

The user sets the environment variable EDITOR so that their default text editor is now nano. However, when a new subshell is created via the bash command, the setting is lost. To keep the setting persistent, it needs to be exported as well as stored in a user's environment file. An example of exporting the variable is shown below.

List: Demonstrating the export command
$ export EDITOR='nano'

It is also important to make sure you have environment variables in the correct file, global or per user. Also, you must understand what type of login the user conducts in order to set these variables in the right file (read this guide)

User issue troubleshooting is tricky. Many components are involved in access, authentication, file management, and the user's shell and environment. Understanding common problems will assist in fixing an issue.

Important exam questions

1. Summarize user access problems/solutions.
For impeded local access, research corrupted terminal files, improperly configured GUI components, and expired passwords/accounts. Remote access problems are often caused by misconfigured OpenSSH components or remote desktop applications. Other issues can involve layered authentication software such as PAM or a system's kernel security module, such as SELinux or AppArmor.

2. Describe various file problems/solutions.
File access and management requires understanding of basic file and directory permissions as well as ownership and group membership. Additional system items to review include filesystem quotas, disk space, inode use, and umask settings. Check the kernel security module log files for policy violations as well. If a user cannot delete a file, look for the immutable bit set on the file.

3. Explain user environment and shell issues.
Improperly configured environment variables or ones that are not exported will cause user problems. Examine the various environment files, both global and user, for issues. Difficulties may also arise from the user account's default shell setting.