Happy New Year! and welcome to a new video series we’re calling “OTel in minute” – bite sized videos with actionable OpenTelemetry information you can use in less than a minute, or thereabouts!
If you’ve ever worked with the OpenTelemetry collector, you’ve probably had questions about the telemetry data coming in and out like of the collector:
Maybe you’ve sent you’re telemetry to the “debug exporter” in the terminal to take a look at, but this can be unwieldy and by definition, only tells you what’s coming out of a pipeline, not at the input or other stages along the way.
In this first video, we’re going to take a look at the “Remote Tap Processor” – a processor you can position any place in your OpenTelemetry Collector pipelines that provides you with a convenient sample of the telemetry flowing through it. This processor is part of the “contrib” repo on Github.
In our case, we’re going to run two Remote Tap Processors, with one at the start and one at the end of our pipelines.
Here’s our collector configuration, you can see two processors defined, one exposing a Websocket on port 12001 (the default) – the start of our pipelines – and one on 12002, for the end of our pipelines, and you can see these processors positioned at the start and end of our logs and traces pipelines here. We’re running our OpenTelemetry collector in a Docker container using this configuration on our localhost, making sure we expose these two ports on the container.
We’re using a handy utility called “Otelgen“ to source our telemetry and another utility called “websocat” to run a webcoket client to receive the sampled telemetry stream in our terminal.
Let’s taking a look at the data coming into our logs pipeline – we’ve piped it to “jq” to help read the JSON better, we can now clearly see the attributes that are being populated – in this case K8s blah
In our configuration, we had a couple of processors to optimize our logs, to filter out low severity logs and deduplicate them respectively.
We can see the logs coming at the start of the pipeline on the left, and the end of the pipeline on the right, and by greping for the severity text, we see that we’re not getting any low severity logs coming out of the pipeline as expected
The deduplication processor adds an attribute to our logs with the number of logs deduplicated, we can see that attribute is being added successfully here and that we’re not getting much in the way of deduplication for this particular log stream
Well that’s today’s “OTel in a minute” for more information and step by step instructions, check out our blog! – speak to you next time!
Dstl8 reads every log line, not just the status code, and tells you what changed, what broke, and what stayed healthy. No dashboards to build, no alert rules to write.
See how Dstl8 worksSilent LLM API errors, Kubernetes networking faults, database timeouts: what they look like in your logs and how to find them before a customer does.
Browse the Incident LibraryJoin engineering teams catching emergent patterns in staging before they page you at 2am.
If you’ve been using Gonzo to pipe logs into your terminal, you already know the basics. But there’s a lot more hiding beneath the surface — AI analysis, attribute search, native Kubernetes support, theming, and integrations with practically every log source in your stack. This post covers the tips and recent enhancements that will change […]
A powerful, real-time log analysis terminal UI inspired by k9s. Analyze log streams with beautiful charts, AI-powered insights, and advanced filtering – all from your terminal. In this scenario, we walk through how Gonzo can make sense of your logs faster. This includes both logs from your applications as well as the Kubernetes control plane […]
