Driftless Agent Configuration Examples

This directory contains example configurations for common Driftless agent use cases. Each example includes all necessary configuration files (agent.yml, apply.yml, facts.yml, logs.yml) for a complete setup.

Available Examples

Basic Monitoring Agent

Files: agent-basic-monitoring.yml, facts-basic-monitoring.yml Purpose: Minimal agent setup for collecting essential system metrics Use Case: Simple infrastructure monitoring without complex logging or configuration enforcement Features:

• CPU, memory, disk, and network monitoring
• Prometheus metrics endpoint
• 1-minute collection intervals

agent-basic-monitoring.yml

# Basic Infrastructure Monitoring Agent Configuration
# This example shows a minimal agent setup for monitoring system metrics

# Agent Configuration
config_dir: "~/.config/driftless/config"
apply_interval: 3600  # Check configuration every hour
facts_interval: 60    # Collect metrics every minute
apply_dry_run: false
metrics_port: 8000
enabled: true

facts-basic-monitoring.yml

# Facts Configuration for Basic Monitoring
# Collects essential system metrics and exposes them via Prometheus

collectors:
  # CPU usage and load averages
  - type: cpu
    interval: 60
    enabled: true

  # Memory usage statistics
  - type: memory
    interval: 60
    enabled: true

  # Disk usage for key mount points
  - type: disk
    interval: 300  # 5 minutes
    paths: ["/", "/var", "/tmp"]
    enabled: true

  # Network interface statistics
  - type: network
    interval: 60
    interfaces: ["eth0", "wlan0"]  # Adjust based on your system
    enabled: true

  # System information (hostname, OS, etc.)
  - type: system
    interval: 3600  # 1 hour
    enabled: true

# Export collected metrics
exporters:
  - type: prometheus
    port: 8000
    path: "/metrics"
    enabled: true

Log Aggregation Agent

Files: agent-log-aggregation.yml, logs-comprehensive.yml Purpose: Comprehensive log collection from multiple sources with forwarding to various destinations Use Case: Centralized logging infrastructure Features:

• Multi-source log collection (nginx, system, application, Docker)
• Multiple output destinations (S3, ELK stack, syslog, local files)
• Compression and batching for efficiency

agent-log-aggregation.yml

# Comprehensive Log Aggregation Agent Configuration
# This example shows how to collect logs from multiple sources and forward them to various destinations

# Agent Configuration
config_dir: "~/.config/driftless/config"
apply_interval: 3600  # 1 hour (reduced since we're focusing on logs)
facts_interval: 300   # 5 minutes (reduced frequency)
apply_dry_run: false
metrics_port: 8001    # Different port to avoid conflicts
enabled: true

logs-comprehensive.yml

# Comprehensive Log Aggregation Configuration
# Collects logs from multiple sources and forwards to various destinations

sources:
  # Web server access logs
  - type: file
    name: nginx-access
    paths:
      - "/var/log/nginx/access.log"
      - "/var/log/nginx/access.log.1"
    parser: common  # Apache Common Log Format
    multiline:
      pattern: '^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}'  # IP address at start
      negate: false
    enabled: true

  # Web server error logs
  - type: file
    name: nginx-error
    paths: ["/var/log/nginx/error.log"]
    parser: nginx_error
    enabled: true

  # System authentication logs
  - type: file
    name: system-auth
    paths: ["/var/log/auth.log", "/var/log/secure"]
    parser: syslog
    enabled: true

  # System messages
  - type: file
    name: system-messages
    paths: ["/var/log/messages", "/var/log/syslog"]
    parser: syslog
    enabled: true

  # Application logs (JSON format)
  - type: file
    name: application-json
    paths: ["/var/log/application/*.log"]
    parser: json
    multiline:
      pattern: '^{\s*"timestamp"'  # JSON objects starting with timestamp
      negate: false
    enabled: true

  # Docker container logs
  - type: file
    name: docker-logs
    paths: ["/var/lib/docker/containers/*/*.log"]
    parser: json
    enabled: true

outputs:
  # Long-term S3 storage with compression
  - type: s3
    name: s3-long-term-storage
    bucket: my-company-logs
    region: us-east-1
    prefix: logs/{{ year }}/{{ month }}/{{ day }}/
    compression:
      algorithm: gzip
      level: 6
    batch:
      max_size: 1000  # 1000 log entries per batch
      max_age: 300    # 5 minutes max batch age
      max_bytes: 5242880  # 5MB max batch size
    enabled: true

  # Real-time ELK stack forwarding
  - type: http
    name: elasticsearch-bulk
    url: http://elasticsearch:9200/_bulk
    method: POST
    headers:
      Content-Type: "application/x-ndjson"
    auth:
      type: basic
      username: elastic
      password: "{{ elasticsearch_password }}"
    batch:
      max_size: 100
      max_age: 30
    timeout: 30
    enabled: true

  # Local syslog for immediate visibility
  - type: syslog
    name: local-syslog
    facility: local0
    severity: info
    tag: driftless
    enabled: true

  # Local file archive for backup
  - type: file
    name: local-archive
    path: "/var/log/driftless/archive"
    rotation:
      max_size: "100MB"
      max_age: "30d"
      max_files: 30
      compress: true
    enabled: true

Configuration Enforcement Agent

Files: agent-config-enforcement.yml, apply-config-enforcement.yml Purpose: Continuous enforcement of system configuration and security policies Use Case: Compliance and security hardening Features:

• Security package installation
• Firewall configuration
• SSH hardening
• System security settings
• Automated security monitoring

agent-config-enforcement.yml

# Configuration Enforcement Agent Configuration
# This example focuses on continuous configuration enforcement and compliance

# Agent Configuration
config_dir: "~/.config/driftless/config"
apply_interval: 600   # Check configuration every 10 minutes
facts_interval: 300   # 5 minutes (reduced since focus is on config)
apply_dry_run: false  # Actually enforce configuration
metrics_port: 8002
enabled: true

apply-config-enforcement.yml

# Configuration Enforcement Tasks
# Ensures system configuration remains compliant with policy

tasks:
  # Security packages
  - type: package
    name: fail2ban
    state: present

  - type: package
    name: ufw
    state: present

  - type: package
    name: auditd
    state: present

  # Security services
  - type: service
    name: fail2ban
    state: started
    enabled: true

  - type: service
    name: ufw
    state: started
    enabled: true

  - type: service
    name: auditd
    state: started
    enabled: true

  # Firewall configuration
  - type: ufw
    rule: allow
    port: "22"
    proto: tcp
    from_ip: "10.0.0.0/8"

  - type: ufw
    rule: allow
    port: "80"
    proto: tcp

  - type: ufw
    rule: allow
    port: "443"
    proto: tcp

  - type: ufw
    rule: default
    policy: deny
    direction: incoming

  # SSH hardening
  - type: file
    path: "/etc/ssh/sshd_config"
    state: present
    content: |
      # SSH configuration enforced by driftless
      PermitRootLogin no
      PasswordAuthentication no
      X11Forwarding no
      MaxAuthTries 3
      ClientAliveInterval 60
      ClientAliveCountMax 3
    mode: "0644"
    backup: true

  - type: service
    name: sshd
    state: restarted

  # System hardening
  - type: sysctl
    name: "net.ipv4.tcp_syncookies"
    value: "1"
    state: present

  - type: sysctl
    name: "net.ipv4.conf.all.rp_filter"
    value: "1"
    state: present

  - type: sysctl
    name: "kernel.randomize_va_space"
    value: "2"
    state: present

  # Log rotation for security logs
  - type: logrotate
    name: auth-log
    path: "/var/log/auth.log"
    options:
      - "weekly"
      - "rotate 12"
      - "compress"
      - "missingok"
      - "notifempty"

  # Cron job for log analysis
  - type: cron
    name: security-log-check
    minute: "0"
    hour: "*/4"
    job: "/usr/local/bin/security-check.sh"
    state: present
    user: root

  # Ensure security monitoring scripts exist
  - type: file
    path: "/usr/local/bin/security-check.sh"
    state: present
    content: |
      #!/bin/bash
      # Security log analysis script
      echo "Running security checks at $(date)"

      # Check for suspicious login attempts
      if grep -q "Failed password" /var/log/auth.log; then
          echo "ALERT: Failed login attempts detected"
          # Send alert here
      fi
    mode: "0755"
    owner: root
    group: root

Production-Ready Agent

Files: agent-production.yml, facts-production.yml, logs-production.yml, apply-production.yml Purpose: Complete production deployment combining monitoring, logging, and configuration enforcement Use Case: Enterprise production environments Features:

• All monitoring capabilities
• Enterprise logging with redundancy
• Comprehensive configuration enforcement
• Security hardening
• Backup and disaster recovery
• TLS encryption for log forwarding

agent-production.yml

# Production-Ready Agent Configuration
# Complete example combining monitoring, logging, and configuration enforcement

# Agent Configuration
config_dir: "/etc/driftless/config"
apply_interval: 1800  # 30 minutes
facts_interval: 60    # 1 minute
apply_dry_run: false
metrics_port: 9090   # Standard Prometheus port
enabled: true

# Environment variables for secrets
secrets:
  aws_access_key_id: "{{ AWS_ACCESS_KEY_ID }}"
  aws_secret_access_key: "{{ AWS_SECRET_ACCESS_KEY }}"
  elasticsearch_password: "{{ ELASTICSEARCH_PASSWORD }}"

facts-production.yml

# Production Facts Configuration
# Comprehensive monitoring setup for production environments

collectors:
  # CPU monitoring
  - type: cpu
    interval: 30
    enabled: true

  # Memory monitoring
  - type: memory
    interval: 30
    enabled: true

  # Disk monitoring for all mount points
  - type: disk
    interval: 300
    paths: ["/", "/var", "/tmp", "/opt", "/home"]
    enabled: true

  # Network monitoring
  - type: network
    interval: 30
    interfaces: ["eth0", "bond0"]  # Production interfaces
    enabled: true

  # Process monitoring
  - type: process
    interval: 60
    processes: ["nginx", "postgres", "redis", "application"]
    enabled: true

  # System information
  - type: system
    interval: 3600
    enabled: true

# Multiple exporters for redundancy
exporters:
  # Primary Prometheus endpoint
  - type: prometheus
    port: 9090
    path: "/metrics"
    enabled: true

  # Backup S3 storage
  - type: s3
    bucket: "{{ metrics_bucket }}"
    region: "{{ aws_region }}"
    prefix: metrics/{{ hostname }}/{{ year }}/{{ month }}/
    interval: 300
    enabled: true

  # Local file backup
  - type: file
    path: "/var/log/driftless/metrics"
    rotation:
      max_size: "50MB"
      max_age: "7d"
      max_files: 7
    enabled: true

logs-production.yml

# Production Logs Configuration
# Enterprise-grade log collection and forwarding

sources:
  # Application logs
  - type: file
    name: application
    paths: ["/var/log/application/*.log"]
    parser: json
    multiline:
      pattern: '^{\s*"timestamp"'
      negate: false
    enabled: true

  # Web server logs
  - type: file
    name: nginx
    paths:
      - "/var/log/nginx/access.log"
      - "/var/log/nginx/error.log"
    parser: nginx_combined
    enabled: true

  # Database logs
  - type: file
    name: postgres
    paths: ["/var/log/postgresql/*.log"]
    parser: syslog
    enabled: true

  # System logs
  - type: file
    name: system
    paths:
      - "/var/log/syslog"
      - "/var/log/auth.log"
      - "/var/log/kern.log"
    parser: syslog
    enabled: true

  # Security logs
  - type: file
    name: audit
    paths: ["/var/log/audit/audit.log"]
    parser: audit
    enabled: true

outputs:
  # Primary ELK stack
  - type: http
    name: elasticsearch
    url: https://elasticsearch.prod.company.com:9200/_bulk
    method: POST
    headers:
      Content-Type: "application/x-ndjson"
    auth:
      type: basic
      username: "{{ elasticsearch_user }}"
      password: "{{ elasticsearch_password }}"
    tls:
      ca_cert: "/etc/ssl/certs/ca.pem"
      client_cert: "/etc/ssl/certs/client.pem"
      client_key: "/etc/ssl/private/client.key"
    batch:
      max_size: 500
      max_age: 60
      max_bytes: 10485760  # 10MB
    retry:
      max_attempts: 3
      backoff: exponential
    enabled: true

  # Backup S3 storage
  - type: s3
    name: s3-backup
    bucket: "{{ logs_bucket }}"
    region: "{{ aws_region }}"
    prefix: logs/{{ hostname }}/{{ year }}/{{ month }}/{{ day }}/
    compression:
      algorithm: gzip
      level: 9
    batch:
      max_size: 2000
      max_age: 600
      max_bytes: 67108864  # 64MB
    enabled: true

  # Local syslog relay
  - type: syslog
    name: local-relay
    facility: local1
    severity: info
    host: "logrelay.prod.company.com"
    port: 514
    protocol: tcp
    enabled: true

  # Emergency local storage
  - type: file
    name: emergency-buffer
    path: "/var/log/driftless/emergency"
    rotation:
      max_size: "1GB"
      max_age: "1d"
      max_files: 7
      compress: true
    enabled: true

apply-production.yml

# Production Configuration Enforcement
# Critical system configuration that must be maintained

tasks:
  # Core system packages
  - type: package
    name: monitoring-tools
    state: present

  - type: package
    name: logrotate
    state: present

  - type: package
    name: unattended-upgrades
    state: present

  # Monitoring services
  - type: service
    name: monitoring-agent
    state: started
    enabled: true

  - type: service
    name: logrotate
    state: started
    enabled: true

  # Security hardening
  - type: sysctl
    name: "net.ipv4.tcp_syncookies"
    value: "1"

  - type: sysctl
    name: "net.ipv4.conf.all.rp_filter"
    value: "1"

  - type: sysctl
    name: "kernel.randomize_va_space"
    value: "2"

  - type: sysctl
    name: "net.ipv4.ip_forward"
    value: "0"

  # SSH hardening
  - type: file
    path: "/etc/ssh/sshd_config.d/driftless.conf"
    state: present
    content: |
      # Production SSH hardening enforced by driftless
      PermitRootLogin no
      PasswordAuthentication no
      MaxAuthTries 3
      ClientAliveInterval 60
      ClientAliveCountMax 3
      AllowTcpForwarding no
      X11Forwarding no
      PermitTTY yes
      PrintLastLog yes
    mode: "0644"

  - type: service
    name: ssh
    state: restarted

  # Log rotation policies
  - type: logrotate
    name: application-logs
    path: "/var/log/application/*.log"
    options:
      - "daily"
      - "rotate 30"
      - "compress"
      - "missingok"
      - "notifempty"
      - "create 0644 {{ application_user }} {{ application_group }}"

  # Backup configuration
  - type: cron
    name: daily-backup
    minute: "0"
    hour: "2"
    job: "/usr/local/bin/backup.sh"
    state: present
    user: backup

  # Monitoring configuration
  - type: file
    path: "/etc/monitoring/agent.yml"
    state: present
    content: |
      # Monitoring agent configuration
      server: monitoring.prod.company.com
      port: 443
      tls: true
      api_key: "{{ monitoring_api_key }}"
    mode: "0600"
    owner: monitoring
    group: monitoring

  # NTP synchronization
  - type: service
    name: systemd-timesyncd
    state: started
    enabled: true

  - type: file
    path: "/etc/systemd/timesyncd.conf"
    state: present
    content: |
      [Time]
      NTP=ntp1.prod.company.com ntp2.prod.company.com
      FallbackNTP=pool.ntp.org
    mode: "0644"

  - type: service
    name: systemd-timesyncd
    state: restarted

  # Disk monitoring
  - type: cron
    name: disk-usage-alert
    minute: "*/15"
    job: "/usr/local/bin/check-disk-usage.sh"
    state: present
    user: root

  # Security updates
  - type: file
    path: "/etc/apt/apt.conf.d/50unattended-upgrades"
    state: present
    content: |
      Unattended-Upgrade::Allowed-Origins {
          "${distro_id}:${distro_codename}-security";
      };
      Unattended-Upgrade::Automatic-Reboot "true";
      Unattended-Upgrade::Automatic-Reboot-Time "02:00";
    mode: "0644"

Configuration File Structure

Each example follows the standard Driftless configuration structure:

~/.config/driftless/
├── config/
│   ├── agent.yml      # Agent behavior configuration
│   ├── apply.yml      # Configuration operations to enforce
│   ├── facts.yml      # Metrics collection configuration
│   └── logs.yml       # Log collection and forwarding configuration
└── data/              # Runtime data (created automatically)

Getting Started

1. Choose an example that matches your use case
2. Copy the configuration files to ~/.config/driftless/config/
3. Edit the configurations to match your environment
4. Set any required environment variables or secrets
5. Start the agent: driftless agent

Environment Variables

Many examples use template variables that should be set as environment variables:

export AWS_ACCESS_KEY_ID="your-key"
export AWS_SECRET_ACCESS_KEY="your-secret"
export ELASTICSEARCH_PASSWORD="your-password"
export MONITORING_API_KEY="your-api-key"

Customization

These examples are starting points. Customize them for your specific needs:

• Adjust collection intervals based on your monitoring requirements
• Modify file paths to match your system layout
• Configure appropriate authentication for external services
• Add additional tasks, collectors, or log sources as needed

Security Considerations

• Store sensitive configuration in environment variables, not in config files
• Use TLS encryption for log forwarding in production
• Implement proper access controls for metrics endpoints
• Regularly rotate credentials and API keys
• Monitor agent resource usage and adjust limits as needed

Troubleshooting

If the agent fails to start:

1. Validate configuration syntax: driftless agent --validate-config
2. Check file permissions on configuration files
3. Verify network connectivity to external services
4. Review agent logs with RUST_LOG=debug driftless agent

For issues with specific components:

• Apply: Check task definitions and system permissions
• Facts: Verify collector configurations and system access
• Logs: Check file paths, permissions, and output destinations