Principles of Information Security: Hashing and Message Digests (SHA, MD5, HMAC)

Hashing and Message Digests (SHA, MD5, HMAC)

Hashing and Message Digests (SHA, MD5, HMAC) – Exam-Ready Study Guide

What This Is

Hashing is a one-way cryptographic function that converts any input (file, password, message) into a fixed-length string of characters (the "hash" or "digest"). Unlike encryption, hashing is irreversible—you can’t retrieve the original data from the hash. Its primary purpose is integrity verification (ensuring data hasn’t been altered) and authentication (e.g., password storage, digital signatures). A real-world example: In 2012, LinkedIn suffered a breach where 6.5 million unsalted SHA-1 password hashes were leaked. Attackers cracked them using rainbow tables, exposing plaintext passwords. This incident highlights why weak hashing (like MD5 or unsalted SHA-1) is dangerous.

Key Terms & Concepts

• Hash Function: A mathematical algorithm that takes an input and produces a fixed-size string (hash). Deterministic (same input → same output) but collision-resistant (hard to find two inputs with the same hash).

• Standards: NIST SP 800-107 (SHA), FIPS 180-4 (Secure Hash Standard).

• Message Digest: Another term for a hash output. Used interchangeably with "hash value."

• SHA (Secure Hash Algorithm): A family of NIST-approved hash functions. Variants:

• SHA-1: 160-bit hash (deprecated due to collision vulnerabilities; e.g., SHAttered attack in 2017).
• SHA-2: Includes SHA-224, SHA-256, SHA-384, SHA-512 (256-bit is most common; used in Bitcoin).

• SHA-3: Latest NIST standard (2015), based on the Keccak algorithm. Not yet widely adopted.

• MD5 (Message Digest Algorithm 5): 128-bit hash. Broken—prone to collisions (e.g., Flame malware used MD5 collisions to forge certificates). Still seen in legacy systems but never use for security.

• HMAC (Hash-Based Message Authentication Code): Combines a hash function (e.g., SHA-256) with a secret key to provide integrity + authenticity. Used in TLS, IPsec, and API authentication.

• Example: HMAC-SHA256(key, message).

• Standard: RFC 2104.

• Salt: Random data added to an input (e.g., a password) before hashing to prevent rainbow table attacks. Stored alongside the hash.

• Example: hash(password + salt).

• Best Practice: Use a unique salt per user (NIST SP 800-63B).

• Pepper: A secret value added to the input (like salt) but not stored with the hash. Used to slow down brute-force attacks.

• Example: hash(password + salt + pepper).

• Collision: When two different inputs produce the same hash. Collision resistance is a key property of secure hash functions.

• Attack Example: Birthday Attack (exploits probability to find collisions faster).

• Preimage Resistance: Hard to reverse a hash to find the original input. Second-preimage resistance means given input1, it’s hard to find input2 where hash(input1) = hash(input2).

• Rainbow Table: A precomputed table of hashes and their plaintext inputs. Used to crack unsalted hashes quickly.

• Defense: Use salting and slow hashing (e.g., bcrypt, Argon2).

• Digital Signature: A hash of a message encrypted with a private key. Proves authenticity + integrity (e.g., code signing, TLS certificates).

• Standard: NIST FIPS 186-5 (DSA, ECDSA).

Step-by-Step / Process Flow

How to Securely Implement Hashing in Practice

1. Choose the Right Algorithm
2. For passwords: Use bcrypt, Argon2, or PBKDF2 (slow, salted hashing).
3. For file integrity: Use SHA-256 or SHA-3.
4. For message authentication: Use HMAC-SHA256.

5. ❌ Never use MD5 or SHA-1 for security.

6. Add Salt (and Pepper if Possible)

7. Generate a random salt (e.g., 16+ bytes) per user.
8. Store the salt in plaintext alongside the hash (e.g., in a database).

9. Example (Python):
   python
import os, hashlib
salt = os.urandom(16) # 16-byte random salt
password = b"my_secure_password"
hash = hashlib.pbkdf2_hmac('sha256', password, salt, 100000)

10. Store Hashes Securely

11. Never store plaintext passwords or unsalted hashes.

12. Use parameterized queries to prevent SQL injection (e.g., when saving hashes to a database).

13. Verify Integrity

14. For files: Compare hashes before/after transfer (e.g., sha256sum file.txt).

15. For passwords: Hash the input + stored salt, then compare with the stored hash.

16. Monitor for Weaknesses

17. Audit systems for MD5/SHA-1 usage (e.g., using tools like Hashcat or John the Ripper).

18. Rotate keys if using HMAC (e.g., every 1–2 years).

19. Respond to Breaches

20. If hashes are leaked, force password resets and upgrade to stronger algorithms.
21. Investigate if the breach involved collision attacks (e.g., forged certificates).

Common Mistakes

Certification Exam Tips

CISSP

• Management Perspective: Focus on risk mitigation (e.g., "Why is SHA-1 deprecated?"). Know NIST standards (FIPS 180-4, SP 800-107).
• Tricky Question: "Which hash function is most resistant to collisions?" → SHA-3 (but SHA-256 is more common in practice).
• HMAC Trap: HMAC provides integrity + authenticity, not confidentiality.

Security+

• Key Distinction: Know the difference between hashing (integrity) and encryption (confidentiality).
• Common Question: "Which algorithm is best for password storage?" → bcrypt or Argon2 (not SHA-256).
• HMAC Use Case: Often tested in TLS, IPsec, or API authentication.

CEH

• Attack Focus: Know collision attacks (MD5/SHA-1), rainbow tables, and birthday attacks.
• Tool Knowledge: Be familiar with Hashcat, John the Ripper, and md5sum/sha256sum.
• Scenario Question: "An attacker uses a precomputed table to crack hashes. What’s the defense?" → Salting.

Quick Check Questions

1. A security team discovers that an application stores passwords using SHA-256 without salt. What is the most critical risk?
2. A) Passwords can be decrypted.
3. B) Rainbow table attacks can crack passwords.
4. C) The hash function is too slow for authentication.

5. D) SHA-256 is vulnerable to collisions.
   Answer: B) Rainbow table attacks can crack passwords.
   Explanation: Unsalted hashes are vulnerable to precomputed rainbow tables, even if SHA-256 is used.

6. Which of the following provides both integrity and authenticity for a message?

7. A) SHA-256
8. B) AES-256
9. C) HMAC-SHA256

10. D) RSA
    Answer: C) HMAC-SHA256.
    Explanation: HMAC combines a hash function with a secret key to ensure the message hasn’t been altered (integrity) and comes from a trusted source (authenticity).

11. During a penetration test, an attacker finds two different files with the same MD5 hash. What type of attack is this?

12. A) Brute-force attack
13. B) Collision attack
14. C) Preimage attack
15. D) Side-channel attack
    Answer: B) Collision attack.
    Explanation: A collision attack finds two different inputs that produce the same hash, exploiting MD5’s weakness.

Last-Minute Cram Sheet

1. Hashing = One-way function (integrity, not confidentiality). ⚠️ Not encryption!
2. SHA-256 = 256-bit hash, NIST-approved (FIPS 180-4). SHA-1/MD5 = broken.
3. HMAC = Hash + secret key (integrity + authenticity). Used in TLS/IPsec.
4. Salt = Random data per user (prevents rainbow tables). Pepper = secret, not stored.
5. Collision = Two inputs → same hash. MD5/SHA-1 are vulnerable.
6. Rainbow table = Precomputed hashes (defense: salt + slow hashing).
7. Password hashing = Use bcrypt/Argon2 (slow), not SHA-256.
8. Digital signature = Hash + private key (proves authenticity).
9. NIST standards: FIPS 180-4 (SHA), SP 800-107 (HMAC).
10. ⚠️ Exam Trap: "Which hash is fastest?" → Wrong question! Slow hashing (bcrypt) is better for passwords.