πŸš€ Observability Stack com OpenTelemetry, Prometheus, Elasticsearch & Grafana (usando Docker)

πŸš€ Observability Stack com OpenTelemetry, Prometheus, Elasticsearch & Grafana (usando Docker)

Modern applications need modern observability. In this guide, you’ll learn how to set up a complete observability stack using Docker Compose, featuring:

  • πŸ›°οΈ OpenTelemetry Collector

  • πŸ“ˆ Prometheus

  • πŸ” Elasticsearch + Kibana

  • πŸ“Š Grafana

πŸ”— Source Code:
The full project is available on GitHub:
github.com/icpmtech/Open-Telemetry-Solutions

Perfect for developers, SREs, and DevOps teams who want logs, metrics, and traces in one place.

🧰 What’s Included?

This stack combines several popular open source tools into a single Docker-based pipeline:

Component Purpose
OpenTelemetry Collector Receives and exports telemetry data
Prometheus Scrapes and stores application metrics
Elasticsearch Stores structured logs and traces
Kibana UI to explore and query Elasticsearch
Grafana Visualizes metrics and logs in dashboards

πŸ“ Project Structure

perl
my-otel-grafana-setup/ β”œβ”€β”€ docker-compose.yml # Launches the full stack β”œβ”€β”€ otel-collector-config.yaml # OpenTelemetry Collector config β”œβ”€β”€ prometheus.yml # Prometheus scrape configuration β”‚ β”œβ”€β”€ elastic_templates/ β”‚ └── otel_template.json # Index template for logs/traces β”‚ β”œβ”€β”€ grafana/ β”‚ └── provisioning/ β”‚ β”œβ”€β”€ datasources/ β”‚ β”‚ β”œβ”€β”€ datasource.yml β”‚ β”‚ └── elasticsearch.yml β”‚ └── dashboards/ β”‚ β”œβ”€β”€ dashboard.yaml β”‚ └── json/ β”‚ └── otel-dashboard.json β”‚ β”œβ”€β”€ infra/ β”‚ β”œβ”€β”€ image.png β”‚ β”œβ”€β”€ main.bicep # Azure Bicep deployment templates β”‚ └── parameters.json β”‚ β”œβ”€β”€ manual-operations/ β”‚ └── configure-elastic.md # Manual steps for Elasticsearch β”‚ └── fastapi-app/ # Optional instrumented example app

πŸ§ͺ Getting Started

1. Clone the Repository

bash
git clone https://github.com/icpmtech/Open-Telemetry-Solutions.git cd Open-Telemetry-Solutions/my-otel-grafana-setup

2. Launch the Stack

bash
docker-compose up -d

This launches all core services.

3. Access the Interfaces

Interface URL Login
Grafana http://localhost:3000 admin / admin
Kibana http://localhost:5601 –
Prometheus http://localhost:9090 –

πŸ“₯ Sending Telemetry Data

You can send telemetry data using:

  • Any app instrumented with OpenTelemetry SDKs

  • The otel-cli tool

OTLP Endpoints

  • Metrics (HTTP): http://localhost:4318/v1/metrics

  • Metrics (gRPC): grpc://localhost:4317

  • Logs: Use the OTLP exporter with fields like @timestamp, trace_id, message

πŸ“Š Preloaded Grafana Dashboard

Grafana comes preloaded with a dashboard that includes:

  • πŸ” Log Stream – Live logs from otel-* indices

  • πŸ“ˆ Time Series Panel – Visualizes log volume over time

πŸ“‚ You’ll find it under: Folder: otel β†’ Dashboard: OpenTelemetry Logs

⚠️ Port Conflicts?

If you get this error:

nginx
Ports are not available: exposing port TCP 0.0.0.0:55679

Fix it by either:

  • Removing the line in docker-compose.yml:

    yaml
    - "55679:55679"

  • Or changing it to an available port:

    yaml
    - "55678:55679"

Then restart the stack:

bash
docker-compose down docker-compose up -d

πŸ› οΈ Extend the Stack

Ideas to grow your observability setup:

  • βž• Add Jaeger for trace visualization

  • πŸ”„ Use Loki instead of Elasticsearch for logs

  • ☁️ Deploy to the cloud using the included Bicep templates

πŸ™Œ Feedback & Contributions

This project is open source β€” feel free to open issues or pull requests with suggestions, bugs, or improvements!

πŸ“„ License

MIT

Back to blog

Our Books

View all

Podcasts

View all
  • ChatGPT Uncovered Podcast

    ChatGPT Uncovered Podcast

    Pedro Martins

    ChatGPT Uncovered Podcast ChatGPT Uncovered Podcast Exploring the Frontiers of AI Conversational Models Episode 1: Understanding ChatGPT Published on: May 15, 2023 Your browser does not support the audio element....

    ChatGPT Uncovered Podcast

    Pedro Martins

    ChatGPT Uncovered Podcast ChatGPT Uncovered Podcast Exploring the Frontiers of AI Conversational Models Episode 1: Understanding ChatGPT Published on: May 15, 2023 Your browser does not support the audio element....

  • Power Apps In-Depth Podcast

    Power Apps In-Depth Podcast

    Pedro Martins

    Power Apps In-Depth Podcast Power Apps In-Depth Podcast Exploring the Capabilities of Microsoft Power Apps Episode 1: Introduction to Power Apps Published on: April 20, 2023 Your browser does not...

    Power Apps In-Depth Podcast

    Pedro Martins

    Power Apps In-Depth Podcast Power Apps In-Depth Podcast Exploring the Capabilities of Microsoft Power Apps Episode 1: Introduction to Power Apps Published on: April 20, 2023 Your browser does not...

  • Exploring Power Pages Podcast

    Exploring Power Pages Podcast

    Pedro Martins

    Exploring Power Pages Podcast Exploring Power Pages Podcast Delving into the World of Microsoft Power Pages Episode 1: Getting Started with Power Pages Published on: March 10, 2023 Your browser...

    Exploring Power Pages Podcast

    Pedro Martins

    Exploring Power Pages Podcast Exploring Power Pages Podcast Delving into the World of Microsoft Power Pages Episode 1: Getting Started with Power Pages Published on: March 10, 2023 Your browser...

1 3
View all