PromCon 2022: why Prometheus had to change

Prometheus continues to hold its own as an essential tool for monitoring, and especially as a key component in observability platforms for cloud native environment. as one of the Cloud-native computing foundation(Fastest-growing CNCF), the time-series database is especially useful for collecting metrics for Kubernetes clusters, and is typically used with Grafana dashboards as an observation tool for visualizations.

Prometheus Metrics and observability data in general have become increasingly essential for many DevOps teams, often overextended and highly distributed in the current pandemic context, especially those working in development environments.

Prometheus installs are now in the hundreds of thousands range with millions of users, Richard (RichiH) Hartmann, Community Director at Grafana Labs and CNCF Technical Advisory Group Observability Chair, said during his talk: “I don’t have to convince this room that Prometheus is a de facto standard in monitoring based on cloud-native metrics.” .

But as Prometheus maintainers celebrate their 10th anniversary, the community’s needs for monitoring Kubernetes are rapidly evolving. Users are also getting smarter about what they want and need. PromCon EU 2022, Held in Munich in November, the annual Prometheus user conference served as a forum on how and why Prometheus should evolve and what Prometheus maintainers should do.

Prometheus Changes

Kubernetes is certainly difficult. Traceability and observability metrics can certainly help control Kubernetes management. But using tools to do that and interpreting often massive amounts of data pose obvious challenges.

In the case of Prometheus, while it gained popularity since former Google engineers originally launched and built Prometheus on SoundCloud in 2012, many found it difficult to use (which Grafana has helped a lot with, as we see below). Thus, usability has been a major issue that Prometheus users have tried to solve during its 10 years of existence, among other challenges the project has faced, Hartmann told The New Stack.

“Prometheus used to have a connotation of being difficult to use,” Hartmann said. “Looking at it today, it’s much easier to get value quickly.”

Much of Prometheus is thanks to its logical and arithmetic operators that with the PromQL query functions meet large-scale data requirements. “The fundamental problem that Prometheus has solved is that, for the first time, it has allowed you to do calculations with your monitoring data in a truly flexible and scalable way,” Hartmann said. “The industry as a whole will never be the same again because people saw what can be done.”

Prometheus is also known to accommodate scaling needs, which is obviously required for drawing inferences from metrics in Kubernetes environments. “We now support a scale that hasn’t been seen before with hyperscalers,” Hartmann said. “No one anticipated that we could handle so much data 10 years ago. It has been by leaps and bounds.”

New stuff

Talks given during PromCon covered new capabilities that users can take advantage of, with the goal of making observability more accessible and powerful for years to come. In other words, the talks delivered didn’t just offer a list of new features and incremental capabilities for what has become the de facto monitoring tool for Kubernetes.

Histograms in Prometheus have had their drawbacks and Grafana’s senior software engineer Ganesh Vernekar‘s in histograms represent one of the most important conference speeches. Histograms in Prometheus have worked reliably for years, but have had some drawbacks when it comes to storage efficiency, histogram query accuracy, and flexibility in histogram usage, Vernekar described. The good news that Vernekar announced during his talk is that Prometheus v2.40.0 now supports native histograms, which is quite a significant advance.

The Prometheus project introduced its Compliance Program last year as a way to help ensure interoperability between data sources promoted as Prometheus-compliant. The program aims to ensure interoperability, protect users from surprises and allow for more parallel innovation, Hartmann said last year.

Over the past year, the testing process and other support offered by the compliance program has matured, making it much easier for organizations to ensure they have performed the proper due diligence. “In essence, it’s very, very easy to say you’re a match. It’s much harder to be truly compliant, particularly when it comes to the nitty-gritty details of how a specific feature behaves,” Hartmann said. “It’s hard for people to figure this out on their own.”

In a nuanced manner, Hartmann described how unproven compatibility claims can cause “confusion,” which is sometimes done “deliberately, sometimes not deliberately,” which is “why we feel compelled to go this route.” Hartman said. As a countermeasure, Hartmann described how the conformance project provides a suite of tests that can be run to test a Prometheus implementation in a cloud environment.

elephant in the room

The elephant in the room during the conference was Grafana. Grafana, of course, has played and continues to play a key role in the development of Prometheus. Grafana added support for Prometheus in 2015 by building its data visualization dashboard to accommodate Prometheus users. Today, it’s hard to find a Prometheus user who doesn’t use Grafana. Grafana continues to maintain its support as a major contributor to open source Prometheus project with more than 44% of Prometheus maintainers coming from Grafana Laboratories.

Grafana has also created a number of its own open source projects for data visualizations, including the recently released Grafana Phlare for continuous profiling and Grafana lighthouse for frontend application observability. These open source projects are based on existing Grafana open source projects. Mimir (metric), Loki (records) Y Tempo (traces).

“Grafana and Prometheus have co-evolved. Grafana, the software, was more suited to Prometheus than vice versa,” Hartmann said. “But Grafana Labs is paying for a substantial amount of the work on Prometheus. Almost symbiotic, although not everyone will like that word.

Cluster Created with Sketch.

Leave a Reply

Your email address will not be published. Required fields are marked *