Troubleshooting Remote Elastic Agent Enrollment in Docker Compose: Common Pitfalls and Solutions

Introduction

If you’ve followed the Elastic Docker Compose tutorial series to set up the Elastic Stack, enrolling remote Elastic Agents might have led to unexpected challenges. Let’s explore a common enrollment failure—DNS resolution errors—and actionable solutions to resolve them.

The Core Issue: DNS and Configuration Conflicts

A user encountered the following error during remote agent enrollment:

plaintext

Copy

lookup fleet-server on 127.0.0.53:53: server misbehaving

This indicates the host couldn’t resolve the fleet-server hostname. Below, we’ll dissect this issue using configuration examples and propose fixes.

Key Configuration Insights

1. Fleet Server Host Configuration

The Fleet Server host is configured with the URL https://fleet-server:8220. While valid within the Docker network, external hosts cannot resolve fleet-server without explicit DNS mapping.

2. Elasticsearch Output Configuration

The Elasticsearch output points to https://es01:9200. Similar to fleet-server, es01 is a Docker service name and won’t resolve externally.

3. APM Integration Settings

The APM configuration includes serverUrl: "https://fleet-server:8200". Note the port 8200 instead of 8220—a common typo. Consistency across configurations is critical.

Step-by-Step Fixes

1. Resolve DNS for Remote Hosts

• Update /etc/hosts on the remote host:
• plaintext
• Copy

# Replace [DOCKER_HOST_IP] with your Docker host's IP

• [DOCKER_HOST_IP]  fleet-server  es01
• Verify connectivity:
• bash
• Copy

ping fleet-server  # Should resolve to Docker host IP

• nc -zv fleet-server 8220  # Check port accessibility

2. Validate Port Exposure in Docker Compose

Ensure ports for fleet-server and es01 are exposed:

yaml

Copy

services:

  fleet-server:

    ports:

      - "8220:8220"  # Expose Fleet Server port

  es01:

    ports:

      - "9200:9200"  # Expose Elasticsearch port

3. Correct APM Configuration

Update the APM serverUrl to use port 8220:

yaml

Copy

serverUrl: "https://fleet-server:8220"

4. Handle Self-Signed Certificates

For testing, bypass certificate validation with --insecure:

bash

Copy

sudo ./elastic-agent install --url=https://fleet-server:8220 --enrollment-token=TOKEN --insecure

Why This Isn’t Production-Ready

The Docker Compose tutorial focuses on local development. For production:

• Avoid hardcoded Docker DNS names: Use domain names or internal DNS.
• Replace self-signed certificates: Deploy trusted certificates (e.g., Let’s Encrypt).
• Secure ports: Restrict access to ports 8220/9200 via firewalls or VPNs.

Production Checklist

1. DNS: Use a dedicated DNS server or map entries on remote hosts.
2. Networking: Ensure ports 8220 (Fleet Server) and 9200 (Elasticsearch) are securely accessible.
3. Certificates: Use trusted SSL certificates for fleet-server and es01.
4. Policies: Start with minimal integrations (e.g., System metrics) to reduce complexity.

Community Feedback

Discussions highlight requests for advanced tutorials covering:

• TLS security for Fleet Server and Elasticsearch.
• Scaling Fleet Server with load balancers.
• Kubernetes integration for dynamic environments.

Final Recommendations

Enrollment failures often stem from:

• DNS misconfigurations: Fix with /etc/hosts or DNS servers.
• Port accessibility: Verify Docker port exposure and firewall rules.
• Certificate trust: Use --insecure for testing, trusted certs for production.

Pro Tip: Test connectivity with nc or curl before enrolling agents!

‍