(NicoElNino/Shutterstock)
To say the oldsters at VictoriaMetrics aren’t large followers of OpenTelemetry after implementing its metrics library could be placing it mildly. “OpenTelemetery is so difficult and bloated,” mentioned Aliaksandr Valialkin, the CTO and co-founder of the observability software program firm.
OpenTelemetry is a Cloud Native Computing Basis (CNCF) venture that was created in 2019 to supply an open supply answer to a thorny laptop downside: to create a unified customary for the three foremost forms of observability knowledge, together with traces, metrics, and logs.
With a single customary, the unique OTel pondering goes, utility builders will acquire entry to raised instruments to correlate several types of observability knowledge with particular servers or digital machines, thereby making it simpler to trace down and repair software program issues. As a bonus, clients would not be locked into proprietary IT monitoring options.
OTel was fashioned by the merger of OpenCensus and OpenTracing, so it shouldn’t be shocking that tracing is essentially the most mature of the three-pronged venture, with the library tracing delivered in 2021. That was adopted in Might 2022 with the preliminary launch of the metrics functionality, whereas the final availability of OTel logging was lastly delivered in November 2023.
A lot of the IT monitoring business, in addition to the three cloud giants, have gotten on board the OTel practice. From established distributors like Splunk and Dynatrace to newer outfits like Elastic and Sumo Logic, supporting OTel has been fashionable throughout the business.
When a few of VictoriaMetrics’s clients requested that the corporate add help for OTel metrics, the corporate obliged. Below the steering of CTO Valialkin, VictoriaMetric builders added the OTel metrics library to its providing, which permits customers to gather, shops, and analyze metrics coming from a variety of various techniques.
Nonetheless, the mixing with OTel metrics didn’t go as deliberate. In accordance with Valialkin, the addition of the metrics library prompted the VictoriaMetrics binary to extend in measurement by 50%, or a number of hundred megabytes.
“It’s an enormous worth for including help for extra ingestion format, like OpenTelemetry,” Valialkin instructed Datanami. “Ultimately, we re-wrote this library from scratch and carried out it mainly solely for VictoriaMetrics. And we’re proud of the end-result as a result of it elevated the binary measurement by lower than 1%, and that’s a lot better than 50% with the official library for OpenTelemetry.”
The OTel library for metrics is so bloated and inefficient as a result of it tries to do an excessive amount of, based on Valialkin. It was designed by a committee to handle the assorted metrics wants of a number of distributors. Nonetheless, the tip product ended up together with a wide variety of “edge instances” that VictoriaMetrics clients will not often, if ever encounter.
A lot of the performance included within the OpenTelemtery customary for metrics isn’t really utilized by any distributors or clients, Valialkin mentioned. “I believe that most individuals don’t cope with the low-level particulars of the OpenTelemetry protocol,” he mentioned. “That’s why they don’t seem to be conscious of this problem which we confronted when including help for OpenTelemetry in VictoriaMetrics.”
(Shutterstock/AI-generated)
In brief, OpenTelemetery for metrics is a “automobile designed by a monkey,” the corporate mentioned.
“Our ache level was that our try to undertake this customary for metric congestion prompted plenty of hassle,” mentioned Roman Khavronenko, a VictoriaMetrics engineer. “It was actually painful to do. It was costly when it comes to engineering hours. It was costly when it comes to high quality of code.”
Others have filed comparable compliants. A consumer on Hacker Information final 12 months mentioned working with OTel was not simple.
“I discover myself having to assume orthogonally to widespread sense each time I attempt to use considered one of its SDKs,” wrote consumer iofiiiiiiiii. “Nothing works the way in which you anticipate it to, every little thing has 3 layers of pointless abstraction and must be approached by way of the again door. Many options have caveats about when it really works, the place it really works, how a lot it really works, throughout what section of the moon it really works and the way lengthy your strings will be when Jupiter is seen within the sky.”
Nonetheless, regardless of the ache, OTel customers nonetheless see the worth of the strategy. Hacker Information consumer iofiiiiiiiii said that the open supply venture is “a partial win.” “Whereas the developer expertise is fairly disagreeable and I’m additionally disenchanted with the precise every day utilization, from an architectural perspective it opens up new alternatives that didn’t exist earlier than.”
Although Valialkin and the VictoriaMetrics group have soured on OTel as a complete, the core thought–to standardize observability knowledge–stays a very good one.
“I really like the thought of OpenTelemtery to unify observability right into a single place so as to have the ability to acquire and rework and course of observability knowledge in a single place,” Valialkin mentioned. “The concept is sweet. However the implementation isn’t superb. We’re not very proud of it.”
A more sensible choice for metrics already exists, Valialkin mentioned. Prometheus has a a lot smaller library, and its textual content exposition protocol is environment friendly, easy, and extensively used. It’s already a de-facto customary, and would make a greater official customary than what the OTel group got here up with, he mentioned.
“It’s extensively utilized by many software program corporations,” Valialkin mentioned. “It has good protection for metrics. It’s simple to debug. It’s simple to implement. It’s simple to parse in comparison with OpenTelemetry. And it isn’t too complicated. It isn’t bloated in comparison with OpenTelemetry.”
Alas, Prometheus is simply used for metrics, and doesn’t help OTel requirements for labeling, which is vital for correlating with different observability knowledge sorts. “It could be laborious to change between logs and traces and metrics when utilizing Prometheus textual content exposition format for metrics assortment,” Valialkin mentioned.
VictoriaMetrics is shifting ahead with its help for the OTel metrics knowledge customary, albeit by its personal customized library. Even with the slimmed down customized library, ingesting OTel metrics knowledge is slower and consumes extra sources than any of the opposite metric knowledge sorts supported by VictoriaMetrics.
“Now we have help of 10 protocols, or one thing like that, for ingestion,” Khavronenko mentioned. “None of them is so complicated.”
In the meantime, VictoriaMetrics is shifting ahead with a brand new product: VictoriaLogs. The corporate can also be adopting OTel’s log protocol for the product, and is girding for extra of the identical.
“I believe that it shares the identical complexity as the usual for metrics,” Valialkin mentioned,” so which may be the primary stumbling level.”
Associated Objects:
OpenTelemetry Beneficial properties Momentum as Observability Normal
Why Roblox Picked VictoriaMetrics for Observability Information Overhaul
Open Supply Instances Collection Database VictoriaMetrics Sees Vital Progress
