Forward Deployed Engineer 101: Ownership and Entrepreneurial Mindset

Ownership and Entrepreneurial Mindset

Ownership & Entrepreneurial Mindset – Field-Ready Study Guide

What This Is

Ownership means treating the customer’s problem as your own—whether it’s a last-minute model deployment in a classified SCIF, debugging a failing pipeline during a hurricane response, or talking a panicked CIO off the ledge during a production outage. An entrepreneurial mindset means thinking like a founder: you’re not just shipping code; you’re delivering outcomes, managing risk, and often improvising with limited resources. Example: You’re on-site at a DoD base when the customer realizes their "urgent" ML model needs to run on a 10-year-old server with no GPU. Instead of saying "not my problem," you rewrite the inference layer in C++ to run on CPU, get it approved by security, and deploy it before the mission briefing—while documenting the trade-offs for future sprints.

Key Terms & Concepts

• End-to-End Ownership: You’re responsible for the entire lifecycle—discovery, design, deployment, monitoring, and deprecation. If the pipeline breaks at 3 AM, you’re the one fixing it, not just the "data team."
• Customer Zero: The first real user of your solution. In FDE work, this is often a stressed-out analyst or a warfighter in the field. Their feedback is your North Star.
• Technical Debt as a Weapon: Not all debt is bad. Sometimes you choose to take on debt (e.g., hardcoding a config for a 48-hour mission) because shipping fast is more important than perfection. The key is tracking it and paying it down later.
• Ask vs. Infer: The customer says, "We need a dashboard to track supply chain delays." You infer: "They actually need real-time alerts when a shipment is 24 hours late, with a way to reroute it." Build the latter.
• The "No Surprises" Rule: Never let the customer be blindsided. If a deployment might cause downtime, tell them before they notice. If a model’s accuracy drops, flag it before they rely on it.
• Improvised Tooling: In restricted environments, you’ll often build one-off tools (e.g., a Python script to parse logs from a proprietary system, or a Bash wrapper for a legacy CLI). Document these like production code.
• ATO (Authority to Operate): The security approval needed to deploy in government/enterprise. Without it, your code is just a prototype. Know the ATO process for your customer’s environment.
• ACO (Authority to Connect): Permission to plug into a network (e.g., SIPRNet, a bank’s internal systems). Often requires physical access, CAC cards, or VPN tokens.
• The "5 Whys" for Root Cause: When debugging, ask "why?" five times to get to the real issue. Example:
• Why did the pipeline fail? → The API timed out.
• Why did the API time out? → The database query was slow.
• Why was the query slow? → No index on the timestamp column.
• Why was there no index? → The schema was auto-generated by an ORM.
• Why did the ORM generate a bad schema? → The team didn’t review the migrations.
• The "Pre-Mortem": Before deploying, ask: "It’s 3 months from now, and this failed spectacularly. What went wrong?" Write down the top 3 risks and mitigate them now.
• The "Minimum Viable Fix": In a crisis, don’t over-engineer. Example: If a dashboard is broken, hardcode the last known good data and add a warning banner instead of rewriting the ETL.
• The "Customer’s Customer": Always think one layer deeper. If you’re building for a SOC analyst, their "customer" is the warfighter relying on their alerts. Build for the end user, not the person signing your PO.

Step-by-Step / Field Process

1. Discovery: Turn the "Ask" into the Real Problem

• Action: Run a 30-minute "5 Whys" session with the customer. Example:
• Customer: "We need a dashboard to track cyber threats."
• You: "Why?" → "Because our analysts are missing critical alerts."
• You: "Why are they missing alerts?" → "Because they’re buried in a sea of false positives."
• You: "Why are there so many false positives?" → "Because the threshold for ‘critical’ is set too low."
• Real Problem: They need a way to dynamically adjust alert thresholds, not a dashboard.
• Tools: Pen and paper, or a shared doc (e.g., Google Docs, Notion). Avoid Jira/Confluence until you’ve aligned on the problem.

2. Scope the "Minimum Viable Fix" (MVF)

• Action: Write a 1-pager with:
• The real problem (from Step 1).
• The MVF (e.g., "A Python script to auto-tune alert thresholds based on historical false positives").
• What’s not in scope (e.g., "No UI, no multi-tenancy, no long-term storage").
• Risks (e.g., "If thresholds are set too high, we might miss real threats").
• Tools: Markdown or a whiteboard. Get the customer to sign off (even if it’s just a Slack thumbs-up).

3. Build for the Worst-Case Environment

• Action:
• Assume no internet, no root access, and ancient hardware.
• Test in a VM that mimics the customer’s environment (e.g., Ubuntu 16.04, no Docker, SELinux enforced).

• Example commands:
  ```bash
  # Check OS version
  cat /etc/os-release

  Check if Docker is installed (it won't be)

  which docker || echo "No Docker, using system Python"

  Test network connectivity (if air-gapped, this will fail)

  ping 8.8.8.8 || echo "Air-gapped, need offline deps"

  Check Python version (might be 2.7)

  python --version ``` - Tools: Vagrant, Packer, or a spare laptop with the customer’s OS image.

4. Deploy with a "No Surprises" Rollout

• Action:
• Before deploying:
  • Write a rollback plan (e.g., "Revert to commit abc123 and restart the service").
  • Send a 24-hour heads-up to the customer: "Deploying at 1400 ET. Expected 5-minute downtime. Rollback plan attached."
• During deployment:
  • Tail logs in real-time: bash tail -f /var/log/myapp.log | grep -i "error\|warn"
  • Have a hotfix ready (e.g., a Python script to patch a config file).
• After deploying:
  • Run a smoke test (e.g., curl localhost:8080/health).
  • Send a follow-up: "Deployed successfully. No errors in logs. Let me know if you see anything odd."

5. Own the Outcome (Even When It’s Not Your Code)

• Action:

• If the system fails, you’re the first responder. Example:

  
  
  • Customer: "The dashboard is blank!"
  • You: ```bash # SSH into the bastion host ssh -i ~/.ssh/customer_key.pem [email protected]

  # Check if the service is running systemctl status myapp.service

  # Tail logs journalctl -u myapp.service -n 50 --no-pager

  # Reproduce the issue curl localhost:8080/api/data | jq . # (if jq is installed) - If it’s a data issue, write a quick script to validate:python # validate_data.py import pandas as pd df = pd.read_csv("customer_data.csv") print(f"Null values: {df.isnull().sum()}") print(f"Duplicates: {df.duplicated().sum()}") ``` - Push a hotfix or roll back, then document the root cause.

6. Document Like Your Job Depends on It (Because It Does)

• Action:
• Write a runbook (e.g., RUNBOOK.md) with:
  • How to deploy/rollback.
  • Common errors and fixes.
  • Who to contact (with phone numbers).
• Example runbook snippet:
  ```markdown
  ## Error: "Connection refused" on port 8080
  • Check if the service is running: systemctl status myapp
  • Check if the port is open: netstat -tulnp | grep 8080
  • If not, restart: sudo systemctl restart myapp
  • If still broken, check logs: journalctl -u myapp -n 100 ```
• Tools: Markdown, GitHub/GitLab wiki, or a shared Google Doc (if the customer doesn’t use Git).

Common Mistakes

FDE Interview / War Story Insights

1. The "Scope Creep" Trap

• Interviewer: "The customer demands a feature that wasn’t in the original scope. How do you respond?"
• What They’re Testing: Can you balance customer needs with delivery risk?
• How to Answer:
• Acknowledge the request: "I understand this is important for your mission."
• Ask for the "why": "Can you help me understand how this fits into your priorities?"
• Propose a trade-off: "We can add this, but it’ll delay the dashboard by 2 weeks. Is that acceptable, or should we prioritize the dashboard and add this in the next sprint?"
• Document the decision: "I’ll send a quick email summarizing this so we’re aligned."

2. The "It Works on My Machine" Nightmare

• War Story: You deploy a model to a classified network, and it crashes on startup. The customer is furious. You realize the model was trained on Python 3.9, but their environment has Python 3.6.
• How to Handle It:
• Don’t blame the customer. Say: "I should have tested this in your environment earlier. Let me fix this now."
• Improvise: Rewrite the model to work with Python 3.6 (e.g., avoid f-strings, use pickle instead of joblib).
• Prevent it next time: Add a pre-deployment checklist: "Verify Python version, OS, and dependencies match the customer’s environment."

3. The "Security vs. Speed" Dilemma

• Interviewer: "You’re on a tight deadline, but the security team says your deployment needs a 2-week ATO review. What do you do?"
• What They’re Testing: Can you navigate bureaucracy without sacrificing the mission?
• How to Answer:
• Ask for a temporary ATO (e.g., "Can we get a 48-hour ATO for testing?").
• Propose a low-risk deployment (e.g., "We’ll deploy to a staging environment first, then promote to prod after ATO").
• Escalate if needed: "This is mission-critical. Can we get a waiver for this sprint?"

4. The "Customer Doesn’t Know What They Want" Scenario

• War Story: You build a dashboard exactly as the customer requested, but they hate it. They say, "This isn’t what we needed."
• How to Handle It:
• Don’t get defensive. Say: "I want to make sure we’re solving the right problem. Can we walk through how you’d use this?"
• Show, don’t tell. Build a quick prototype (e.g., a Jupyter notebook with fake data) and say: "Is this closer to what you need?"
• Iterate fast. Ship a new version in 24 hours with their feedback.

Quick Check Questions

1. You’re deploying to an environment where you can’t run standard Docker images due to security restrictions. What’s your first step?

• Answer: Check if Podman or rkt is installed (lightweight container runtimes allowed in some secure environments). If not, build a static binary or use the system’s native package manager (e.g., yum install).
• Why: Docker is often blocked in secure environments, but alternatives like Podman (Docker-compatible) or static binaries are allowed.

2. The customer’s data pipeline fails at 2 AM, and the on-call engineer says, "It’s not my code, so I can’t help." What do you do?

• Answer: SSH into the box, tail the logs, and diagnose the issue. Even if it’s not your code, you own the outcome. Example: bash ssh user@pipeline-server tail -n 50 /var/log/pipeline.log # If it's a Python error, check the stack trace: python -c "import traceback; print(traceback.format_exc())"
• Why: In the field, there’s no "not my problem." You’re the FDE—fix it or find someone who can.

3. You’re on-site, and the customer says, "We need this feature by tomorrow, but it wasn’t in the original scope." How do you respond?

• Answer: Say: "I’ll make this work, but let’s align on trade-offs. Adding this will delay [X] by [Y] days. Is that acceptable, or should we prioritize [X] and add this in the next sprint?"
• Why: You’re not saying no—you’re managing expectations and documenting the decision.

Last-Minute Cram Sheet

1. Always test in the customer’s environment. ⚠️ What works in your lab will break behind their firewall.
2. The "5 Whys" beats the "ask." Dig deeper to find the real problem.
3. ATO = Authority to Operate. No ATO? No deployment.
4. ACO = Authority to Connect. No ACO? No network access.
5. Rollback plan > hotfix. Always have a way to revert.
6. Document like your job depends on it. Because it does.
7. Customer Zero is your North Star. Build for the end user, not the PO.
8. Technical debt is a tool. Use it wisely, pay it down later.
9. No surprises. Tell the customer before they notice.
10. Improvise. In the field, you’ll often build one-off tools (e.g., a Python script to parse logs from a proprietary system). Document them like production code.