Are you prepared to begin your journey on the street to accumulating telemetry knowledge out of your purposes?
On this sequence, you may discover tips on how to undertake OpenTelemetry (OTel) and tips on how to instrument an utility to gather tracing telemetry. You may learn to leverage out-of-the-box automated instrumentation instruments and perceive when it is necessary to discover extra superior guide instrumentation in your purposes. By the top of this sequence, you may have an understanding of how telemetry travels out of your purposes to the OpenTelemetry Collector, and be able to carry OpenTelemetry to your future tasks. Every thing mentioned right here is supported by a hands-on, self-paced workshop authored by Paige Cruz.
The earlier article zoomed into the utilization of Jaeger to assist builders when visually exploring their telemetry knowledge. On this article, we’ll assist builders with manually instrumenting so as to add metadata particular to their utility or enterprise permitting them to derive higher insights sooner.
It’s assumed that you simply adopted the earlier articles in establishing each OpenTelemetry and the instance Python utility challenge, but when not, return and see the earlier articles as it isn’t coated right here.
Thus far on this sequence, we have used automated and programmatic instrumentation to gather tracing knowledge for our system interactions. It could be rather more attention-grabbing to manually instrument our utility, including metadata particular to our enterprise that helps derive higher insights sooner.
Including Span Attributes
Including span attributes means we’re monitoring key, worth pairs containing metadata to annotate a span with details about the operation it’s monitoring. For instance, to set this up for our Python utility, we use the syntax:
span.set_attribute("KEY","VALUE")
To use this to our utility (that you simply put in and used within the earlier articles from this sequence), we open the file guide/app.py, import the get_current_span
from the API, and add an attribute monitoring the depend of homepage masses to the index()
technique as follows with the brand new code proven in daring:
... from opentelemetry import hint from opentelemetry.hint import set_tracer_provider, get_current_span from opentelemetry.sdk.hint import TracerProvider from opentelemetry.sdk.hint.export import SimpleSpanProcessor from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter ... @app.route("https://dzone.com/") def index(): span = hint.get_current_span() world HITS HITS = HITS + 1 span.set_attribute("hits", HITS) msg = f'This webpage has been seen {HITS} instances' return msg ...
We now can construct a brand new container picture and run the pod to confirm as follows:
$ podman construct -t hello-otel:guide -f guide/Buildfile-manual $ podman play kube guide/app_pod.yaml
With the pod working open a browser to http://localhost:8080 and open the Jaeger UI at http://localhost:16686, looking for traces of the operation /
which ought to look one thing like this:
Click on on a hint and ensure the hits
attribute which is in span particulars:
This verifies that the guide instrumentation with its attributes is working for tracing our utility homepage. What about tracing nested spans?
Let’s strive manually instrumenting for nested spans. However first, we have to cease the working pod with the next:
$ podman play kube guide/app_pod.yaml --down
Now let’s improve our monitoring of nested spans.
Enhancing Programmatic Instrumentation
The instrumented requests library offers knowledge about requests to the Canine API in our utility, however not customized strategies get_breed()
or validate_breed()
. To seize that work in /doggo traces we will manually instrument nested spans to mirror their hierarchical relationship. The next creates a brand new span inside the present hint context:
tracer.start_as_current_span()
So as to add this, open guide/app.py and get entry to the world tracer. That is what really creates and manages spans, proven beneath in daring for the brand new code:
... app = Flask("hello-otel") FlaskInstrumentor().instrument_app(app) Jinja2Instrumentor().instrument() RequestsInstrumentor().instrument() tracer = supplier.get_tracer(app.title) ...
Additional down the file, we discover the get_breed()
technique and create a nested span as follows:
... def get_breed(url): with tracer.start_as_current_span("get_breed"): path = urllib3.util.parse_url(url).path match = re.search(r"/breeds/([^/]+)/", path) if match: end result = match.group(1) return end result ...
We now must rebuild the container picture and run the pod to confirm as follows:
$ podman construct -t hello-otel:guide -f guide/Buildfile-manual $ podman play kube guide/app_pod.yaml
With the pod working open a browser to http://localhost:8080/doggo and make a number of requests by refreshing the web page. Open the Jaeger UI at http://localhost:16686, looking for traces for the operation /doggo. It’s best to see that the span depend has elevated however to substantiate, click on on a hint:
Now let’s confirm we’re receiving tracing knowledge from the nested spans. The hint waterfall ought to present one span for the general request to /doggo, the GET
request to the Canine API, and the operation get_breed()
to present a fuller image of the place time is spent fulfilling requests to /doggo:
To this point we have used the hint waterfall to visualise our traces. Let’s strive an alternative choice by clicking on the menu within the higher proper nook and choosing Hint Graph:
By clicking on the T over on the right-hand vertical menu bar, we will shade the crucial path or the spans that immediately contribute to the slowest path of processing this request. That is how we will view a hint graph by time:
On this instance, we’re solely working with small traces. You possibly can think about with bigger hint samples, corresponding to this instance with 160 spans, a birds-eye view generally is a very highly effective instrument when troubleshooting extra advanced traces:
Now let’s strive including extra span occasions to our utility, however first, we have to cease the working pod so we will add extra code to the appliance:
$ podman play kube guide/app_pod.yaml --down
Now let’s add span occasions.
Including Span Occasions
An occasion incorporates a structured log with a title, a number of attributes, and a timestamp. These are used so as to add structured annotations to a present span:
span.add_event(title, attributes)
So as to add one to our utility, open the file guide/app.py and add a span occasion representing the results of the cube roll to the roll_dice()
technique as follows. Be aware the brand new code is in daring sort:
... @app.route("/rolldice") def roll_dice(): sp = hint.get_current_span() end result = do_roll() sp.add_event("rolled dice",attributes={"result":end result}) return end result ...
We now must rebuild the container picture and run the pod to confirm as follows:
$ podman construct -t hello-otel:guide -f guide/Buildfile-manual $ podman play kube guide/app_pod.yaml
With the pod working, open a browser to http://localhost:8080/rolldice and make a number of requests by refreshing the web page. Open the Jaeger UI at http://localhost:16686, seek for traces for the operation /rolldice, and click on on a hint:
It’s of notice that the timestamp related to this span occasion is relative to the beginning time of the hint itself:
Now let’s strive exploring span standing, however first, we have to cease the working pod so we will add extra code to the appliance:
$ podman play kube guide/app_pod.yaml --down
Now let’s examine what defines a span standing.
Including Span Standing
A span standing is often used to determine spans that haven’t been accomplished efficiently and may be set any time earlier than the span is completed. Spans can have a standing of Unset, OK, or Error.
span.set_status(standing, description)
The /doggo web page permits customers to seek for pictures of a selected canine breed. If the search time period is invalid, an error message is returned to the person and the request efficiently returns 200 OK
. If we take into account invalid searches to be errors, we have to instrument validate_breed()
to hint this exercise. This is not going to make the complete hint failed
, however noting the error on a span will probably be useful to our utility visibility.
So as to add a span standing, open the file guide/app.py and find validate_breed()
. Create a nested span and add an attribute for the breed to assist us perceive what the person was looking for as follows with daring sort indicating our new code:
... def validate_breed(breed): with tracer.start_as_current_span("validate_breed") as span: span.set_attribute("breed", breed) if breed not in breeds: increase ValueError("No breed found.") return ...
We now must rebuild the container picture and run the pod to confirm as follows:
$ podman construct -t hello-otel:guide -f guide/Buildfile-manual $ podman play kube guide/app_pod.yaml
With the pod working, open a browser to http://localhost:8080/doggo and make a number of requests by refreshing the web page. Seek for legitimate canine breeds like Doberman or Akita and nonsense phrases like woof:
Now open the Jaeger UI at http://localhost:16686 and seek for traces for the operation /doggo. We’re verifying that there’s at the very least one hint with a profitable request marked with an error. Click on that hint to see the detailed view:
Confirm that the error message is recorded on the search_breed
span and that the breed is recorded as a span attribute:
As now we have seen, combining programmatic and guide instrumentation to boost our utility visibility supplied by tracing knowledge is one other highly effective addition to our toolbox. We have explored and added span attributes, enhanced our programmatic instrumentation, added span occasions, and explored tracing span standing.
These examples use code from a Python utility that you may discover within the supplied hands-on workshop.
What’s Subsequent?
This text detailed manually instrumenting our utility so as to add metadata particular to our enterprise wants, permitting us to derive higher insights sooner.
In our subsequent article, we will probably be turning to manually instrumenting metrics for our utility.