Datadog Agent Setup for Python App on EC2

Purpose: Install and configure the Datadog Agent to collect host metrics and APM traces from a Python FastAPI application. This setup works for autoscaling environments and ensures host metrics and traces are reported automatically.

1. Prerequisites

• EC2 instance with Python 3.8+ and a virtual environment (venv) installed.
• Datadog account with API key.
• Security group allowing outbound HTTPS (port 443) for agent reporting.
• Python app installed at /home/<user>/<app-name> (example uses example-app).

2. Install Datadog Agent

1. Set environment variables for your Datadog API key, site, and environment:

export DD_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
export DD_SITE="datadoghq.com"   # Use datadoghq.eu for EU, datadoghq.in for India
export DD_ENV=prod

2. Install Datadog Agent v7:

bash -c "$(curl -L https://install.datadoghq.com/scripts/install_script_agent7.sh)"

3. Verify installation:

sudo systemctl status datadog-agent
sudo datadog-agent status

3. Configure Datadog Agent

1. Edit the main configuration file:

sudo nano /etc/datadog-agent/datadog.yaml

Ensure the following entries exist:

api_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
site: datadoghq.com
tags:
  - env:prod
 
apm_config:
  enabled: true
  env: prod

2. Restart the agent:

sudo systemctl restart datadog-agent

3. Verify APM listener:

sudo datadog-agent status | grep -A10 "APM Agent"

• Should show listener running on 127.0.0.1:8126.

4. Python Application Setup (example-app)

Directory structure example:

/home/ec2-user/example-app/
├── venv/
├── main.py
├── requirements.txt

1. Activate virtual environment and install dependencies:

source /home/ec2-user/example-app/venv/bin/activate
pip install ddtrace uvicorn fastapi

5. Create Systemd Service for Python App

1. Create the unit file /etc/systemd/system/example-app.service:

[Unit]
Description=Example Python FastAPI Application
After=network.target
 
[Service]
User=root
Group=root
WorkingDirectory=/home/ec2-user/example-app
Environment=DD_AGENT_HOST=127.0.0.1
Environment=DD_SERVICE=example-app
Environment=DD_ENV=prod
Environment=DD_VERSION=1.0.0
Environment=DD_TRACE_REMOVE_INTEGRATION_SERVICE_NAMES=true
ExecStart=/home/ec2-user/example-app/venv/bin/ddtrace-run /home/ec2-user/example-app/venv/bin/uvicorn main:app --host 0.0.0.0 --port 8000
Restart=always
RestartSec=5
StandardOutput=journal
StandardError=journal
 
[Install]
WantedBy=multi-user.target

2. Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable example-app
sudo systemctl start example-app
sudo journalctl -u example-app -f

ddtrace-run ensures traces from the Python app are sent to the local Datadog Agent. Logs are captured in the systemd journal and can be viewed using journalctl.

6. Verification

1. Host Metrics:

   • Go to Datadog Dashboard → Host Metrics
   • Verify CPU, memory, disk, and network metrics from the EC2 instance.

2. APM Traces:

   • Go to Datadog APM → Tracing Dashboard
   • Requests to your FastAPI app should appear automatically with traces.

3. Agent Status:

   sudo datadog-agent status

   Ensure:

   • Forwarder API key is valid.
   • Core checks (system.cpu, system.mem, etc.) show OK.
   • APM Agent listener is running.

✅ References

• Datadog Agent Installation for Linux (opens in a new tab)
• Datadog Custom Dashboards (opens in a new tab)
• Datadog APM Tracing (opens in a new tab)