## Intro to Prometheus With a dash of operations & observability ## Richard Hartmann & Frederic Branczyk @TwitchiH & @fredbrancz 2018-12-12 ## Who are we? ## Richard "RichiH" Hartmann DENOGx, PromCon staff Prometheus team member ## Frederic Branczyk • Red Hat (previously CoreOS) • All things Prometheus / Kubernetes • Kubernetes SIG-Instrumentation lead Prometheus team member ## Time split 1/3 Prometheus 2 1/3 Observability 3 1/3 Questions ## Show of hands Who has heard of Prometheus? Who is considering to use Prometheus? Who is POCing Prometheus? Who uses Prometheus in production

### Prometheus Deep Dive Monitoring. At scale Richard Hartmann & Frederic Branczyk @TwitchiH & @fredbrancz 2018-12-12 ## Who are we? ## Richard "RichiH" Hartmann • Swiss army chainsaw PromCon staff • Prometheus team member ## Frederic Branczyk • Red Hat (previously CoreOS) • All things Prometheus / Kubernetes • Kubernetes SIG-Instrumentation lead • Prometheus team member ## Show Show of hands Who has heard of Prometheus? Who is considering to use Prometheus? Who is POCing Prometheus? Who uses Prometheus in production? ## Prometheus 101 • Inspired by Google’s Borgmon • Time

## RUST CHINA CONF 2023 第三届中国 Rust 开发者大会 6.17-6.18 @Shanghai ## PromQL Got a BOOST: 用 Rust 重写 Prometheus 的查询引擎 Ruihang Xia @greptime.com ## I ABOUT ## Ruihang GitHub: waynexia Losing hair at Greptime Background ## About Prometheus and PromQL Why rewrite? What are boosted? ## PROMETHEUS AND PROMQL  ## Prometheus The de facto alert, visualize with many integrations ## PROMETHEUS AND PROMQL  ## Prometheus rate(api_request_duration{host="foo"}[5m])

--host=cof-lee.com --username=admin --password=xx ## ★第 15 章、prometheus 监控系统 ## ★Prometheus 简介 Prometheus 是一款基于时序数据库的开源监控告警系统，Prometheus 的基本原理是通过 HTTP 协议周期性抓取被监控组件的状态，任意组件只要提供对应的 HTTP 接口就可以接入监控。不需要任何 非常有意义 支持 HTTP 的 Pull 方式采集时间序列数据 支持 PushGateway 采集瞬时任务的数据 支持服务发现和静态配置两种方式发现目标 支持接入 Grafana Prometheus Server 三个组件 ① Retrieval: 数据采集，获取被监控对象的相关数据 ② TSDB: 时间序列数据库（Time Series Database）用来处理时间序列数据的数 语句查询指标信息，并在页面展示。虽然 Prometheus 自带 UI 界面，但是大多数是使用 Grafana 进行绘图展示。另外也可通过 API 接口来给第三方软件提供监控指标。alertmanager 组件负责邮件报警。 ## ★部署 Prometheus 部署清单文件下载：https://github.com/prometheus-operator/kube-prometheus Operator 部署器是根据已经编写好的

TOC • Old-school monitoring • Solving issues in a new way • Monitoring your service mesh ## Prometheus + Kubernetes - A time series based monitoring system. • Borgmon for mere mortals. - Seamless level. - Key - value data model with powerful PromQL. Emerging open source community. ## + ### Prometheus 2.0 - deal with dynamic nature • Storage engine redesign & reimplementation ☐ Save more CPU ## Prometheus at scale ## • In the old days ☐ one or more prometheus per cluster ☐ hashmod sharding  ## Prometheus at

## kubernetes  Prometheus   ## Prometheus ## 2018 -2020 ## 2020 年指标数据 ![Image](/uploads/documents/1/5/2/3/1523709684bb870a419cdaef794bf385/p14_1

Debian Developer Prometheus team member Prometheus ## What's Prometheus? You can’t talk about OpenMetrics without mentioning Prometheus Prometheus ## What's Prometheus? Show of hands: Who Who has heard of Prometheus? Problem statement ## Before Prometheus Historically, the monitoring landscape has been highly fragmented Many solutions based on ancient technology • Most data formats are Problem statement ## Before Prometheus Only thing approaching a standard: SNMP Who has heard of SNMP? Who likes SNMP? XKCD 927 applies ## After Prometheus Prometheus has become a de-facto standard

[Image](/uploads/documents/b/2/d/7/b2d703128539524d374d63151f676cb3/p1_1.jpg) 监控遇上 Kubernetes Prometheus 优化 VictoriaMetrics 实践 性能优化 ## 监控遇上 Kubernetes YYMS 是公司内通用的监控报警系统，有完善的数据收集，展示和报警机制，但是 YYMS kubernetes_sd_config 自动发现服务 通过remote_write协议将数据写至kafka，容器云平台通过消费kafka topic获取容器基础监控数据。容器云平台定制告警规则发送给相应的服务负责人 ● prometheus通过定期执行集群级别的告警规则，将触发告警的信息发送至alertmanger，alertmanager通过webhook将告警信息发送至YYMS • grafana 可视化展现  ## 内存优化 降低样本数量 - 查看当前每秒的样本数 rate(prometheus_tsdb_head_samples_appended_total[1h])

eu-west-1.compute.internalnodeTrue ## Monitoring Prometheus autodiscovery Grafana dashboards Graylogs search and dashboards  ## Conclusion • Documentation and procedures Prometheus • Scaling • Best practice Locust ## Thank you for the attention Any questions? ![Image](/