otel-collector

---
name: otel-collector
description: Expert guidance for configuring and deploying the OpenTelemetry Collector. Use when setting up a Collector pipeline, configuring receivers, exporters, or processors, deploying a Collector to Kubernetes or Docker, or forwarding telemetry to Dash0. Triggers on requests involving collector, pipeline, OTLP receiver, exporter, or Dash0 collector setup.
license: Apache-2.0
metadata:
  author: dash0
  version: '1.0.0'
  workflow_type: 'advisory'
  supports-traces: "true"
  supports-metrics: "true"
  supports-logs: "true"
---

# OpenTelemetry Collector configuration guide

Expert guidance for configuring and deploying the OpenTelemetry Collector to receive, process, and export telemetry.

## Rules

| Rule | Description |
|------|-------------|
| [receivers](./rules/receivers.md) | Receivers — OTLP, Prometheus, filelog, hostmetrics |
| [exporters](./rules/exporters.md) | Exporters — OTLP/gRPC to Dash0, debug, authentication |
| [processors](./rules/processors.md) | Processors — memory limiter, resource detection, ordering, sending queue |
| [pipelines](./rules/pipelines.md) | Pipelines — service section, per-signal configuration, connectors |
| [deployment](./rules/deployment.md) | Deployment — agent vs gateway patterns, deployment method selection |
| [dash0-operator](./rules/deployment/dash0-operator.md) | Dash0 Kubernetes Operator — automated instrumentation, Collector management, Dash0 export |
| [collector-helm-chart](./rules/deployment/collector-helm-chart.md) | Collector Helm chart — presets, modes, image selection |
| [opentelemetry-operator](./rules/deployment/opentelemetry-operator.md) | OpenTelemetry Operator — Collector CRD, auto-instrumentation, sidecar |
| [raw-manifests](./rules/deployment/raw-manifests.md) | Raw Kubernetes manifests — DaemonSet, Deployment, RBAC, Docker Compose |
| [aws-lambda](./rules/deployment/aws-lambda.md) | AWS Lambda Collector layer — configuration layer, `telemetryapi` receiver, `decouple` processor |
| [sampling](./rules/sampling.md) | Sampling — head, tail, load balancing |
| [red-metrics](./rules/red-metrics.md) | RED metrics — span-derived request rate, error rate, duration histograms |
| [custom-distributions](./rules/custom-distributions.md) | Custom distributions — building a stripped-down Collector binary with OCB |

## Key principles

- **Processor ordering matters.**
  Place `memory_limiter` first in every pipeline.
  Enable exporter batching with `sending_queue: { batch: {} }` and persist the queue with `file_storage` instead of using the `batch` processor.
  Incorrect ordering causes memory exhaustion or data loss.
- **One pipeline per signal type.**
  Define separate pipelines for traces, metrics, and logs.
  Mixing signals in a single pipeline breaks processing and causes runtime errors.
- **Every declared component must appear in a pipeline.**
  The Collector ignores components that no pipeline references, without an error or warning, so an unwired component does nothing.
- **Consistent resource enrichment across pipelines.**
  Apply processors that enrich resource attributes like `resource_detection` and `k8s_attributes` to every signal pipeline (traces, metrics, and logs), not just one.
  If one pipeline enriches telemetry with `k8s.namespace.name` or `host.name` but another does not, correlation between signals is compromised by incomplete metadata.
- **Use canonical component names.**
  Write `otlp_grpc`, `otlp_http`, `load_balancing`, `span_metrics`, `log_dedup`, `signal_to_metrics`, `k8s_attributes`, and `resource_detection` as config keys.
  The older names `otlp` (exporter only), `otlphttp`, `loadbalancing`, `spanmetrics`, `logdedup`, `signaltometrics`, `k8sattributes`, and `resourcedetection` still work as deprecated aliases and log a deprecation warning at startup.
- **Run Collector v0.153.0 or later.**
  The configs in this skill need it: `load_balancing` and `resource_detection` exist only from v0.153.0, and `sending_queue.batch: {}` only from v0.138.0.
  An older Collector rejects them at startup: `unknown type` for a new name, and for `batch: {}` either `'sending_queue' has invalid keys: batch` or `` `batch` supports only `items` or `bytes` sizer ``.
  When you pin an image, pin the version the examples are written against, v0.163.1.
- **Memory safety is non-negotiable.**
  Always configure `memory_limiter` in production.
  Without it, a burst of telemetry can cause the Collector to OOM and crash.

## Quick start

Minimal working configuration: OTLP receiver → memory limiter → OTLP/gRPC exporter to Dash0.

```yaml
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 400
    spike_limit_mib: 100

exporters:
  otlp_grpc:
    endpoint: ingress.eu-west-1.aws.dash0.com:4317
    headers:
      Authorization: "Bearer ${env:DASH0_TOKEN}"
    sending_queue:
      enabled: true
      storage: file_storage
      batch: {}

extensions:
  file_storage:
    directory: /var/lib/otelcol/queue
    create_directory: true

service:
  extensions: [file_storage]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter]
      exporters: [otlp_grpc]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter]
      exporters: [otlp_grpc]
    logs:
      receivers: [otlp]
      processors: [memory_limiter]
      exporters: [otlp_grpc]
```

The Collector user must be able to write to the `file_storage` directory; in a container, mount a persistent volume at `/var/lib/otelcol`.

See [exporters](./rules/exporters.md) for full authentication and queue configuration, and [processors](./rules/processors.md) for adding resource detection.

## Configuration workflow

1. **Write config** — define receivers, processors, and exporters; wire them in `service.pipelines`.
2. **Validate locally** — run `otelcol validate --config=config.yaml` to catch structural errors before deployment.
3. **Deploy** — choose a deployment method from the [deployment](./rules/deployment.md) rule (Helm, Operator, raw manifests, or Docker Compose).
4. **Verify** — add the `debug` exporter to a pipeline temporarily and inspect stdout to confirm telemetry is flowing; then remove it before going to production.

## Quick reference

| What do you need? | Rule |
|--------------------|------|
| Accept OTLP telemetry from applications | [receivers](./rules/receivers.md) |
| Scrape Prometheus endpoints | [receivers](./rules/receivers.md) |
| Collect log files or host metrics | [receivers](./rules/receivers.md) |
| Send telemetry to Dash0 | [exporters](./rules/exporters.md) |
| Configure retry, queue, or compression | [exporters](./rules/exporters.md) |
| Set processor ordering | [processors](./rules/processors.md) |
| Add Kubernetes or cloud metadata | [processors](./rules/processors.md) |
| Wire receivers → processors → exporters | [pipelines](./rules/pipelines.md) |
| Complete working configuration | [pipelines](./rules/pipelines.md) |
| Validate the pipeline with the debug exporter | [collector-helm-chart](./rules/deployment/collector-helm-chart.md), [opentelemetry-operator](./rules/deployment/opentelemetry-operator.md), [raw-manifests](./rules/deployment/raw-manifests.md), or [dash0-operator](./rules/deployment/dash0-operator.md) |
| Deploy as DaemonSet or Deployment | [raw-manifests](./rules/deployment/raw-manifests.md) |
| Deploy with Helm | [collector-helm-chart](./rules/deployment/collector-helm-chart.md) |
| Deploy with the OTel Operator | [opentelemetry-operator](./rules/deployment/opentelemetry-operator.md) |
| Deploy with the Dash0 Operator | [dash0-operator](./rules/deployment/dash0-operator.md) |
| Auto-instrument applications in Kubernetes | [opentelemetry-operator](./rules/deployment/opentelemetry-operator.md) or [dash0-operator](./rules/deployment/dash0-operator.md) |
| Local development with Docker Compose | [raw-manifests](./rules/deployment/raw-manifests.md) |
| Run a Collector inside an AWS Lambda function | [aws-lambda](./rules/deployment/aws-lambda.md) |
| Reduce trace volume | [sampling](./rules/sampling.md) |
| Keep errors and slow traces, drop the rest | [sampling](./rules/sampling.md) |
| Redact sensitive data in the pipeline | [processors](./rules/processors.md#sensitive-data-redaction) |
| Generate RED metrics from traces | [red-metrics](./rules/red-metrics.md) |
| Build a custom Collector binary | [custom-distributions](./rules/custom-distributions.md) |

## Official documentation

- [OpenTelemetry Collector documentation](https://opentelemetry.io/docs/collector/)
- [Collector configuration](https://opentelemetry.io/docs/collector/configuration/)
- [Collector contrib components](https://github.com/open-telemetry/opentelemetry-collector-contrib)
- [Dash0 Integration Hub](https://www.dash0.com/hub/integrations)