Introduction Background Operations & observability Outro Intro to Prometheus With a dash of operations & observability Richard Hartmann & Frederic Branczyk @TwitchiH & @fredbrancz 2018-12-12 Richard Richard Hartmann & Frederic Branczyk @TwitchiH & @fredbrancz Intro to Prometheus Introduction Background Operations & observability Outro Who are we? Richard ”RichiH” Hartmann Swiss army chainsaw at SpaceNet DebConf, DENOGx, PromCon staff Prometheus team member Frederic Branczyk Red Hat (previously CoreOS) All things Prometheus / Kubernetes Kubernetes SIG-Instrumentation lead Prometheus team member Richard Hartmann

Outro Prometheus Deep Dive Monitoring. At scale. Richard Hartmann & Frederic Branczyk @TwitchiH & @fredbrancz 2018-12-12 Richard Hartmann & Frederic Branczyk @TwitchiH & @fredbrancz Prometheus Deep DebConf, DENOGx, PromCon staff Prometheus team member Frederic Branczyk Red Hat (previously CoreOS) All things Prometheus / Kubernetes Kubernetes SIG-Instrumentation lead Prometheus team member Richard Hartmann @fredbrancz Prometheus Deep Dive Introduction Intro 2.0 to 2.2.1 2.4 - 2.6 Beyond Outro Show of hands Who has heard of Prometheus? Who is considering to use Prometheus? Who is POCing Prometheus? Who uses

PromQL 从⼊⻔到精通 对于 Prometheus 生态的监控系统，PromQL 是必备技能，本文着重点讲解这个查询语言，掺 杂一些生产实践场景，希望对你有所帮助。 ? 本文作者：秦晓辉，Open-Falcon、Nightingale 等开源项目创始人之一，极客时间《运 维监控系统实战笔记》作者；当前在创业（快猫星云联创），为客户提供监控/可观测性产品 方案，有需求的朋友欢迎联系我的微信 方案，有需求的朋友欢迎联系我的微信 picobyte。 数据类型 Prometheus 有四种数据类型：Gauge、Counter、Histogram、Summary，其中最关键的是 Gauge 和 Counter，Histogram 和 Summary 只是为了上报监控数据的 Client 侧的便利，可 以看做是组合使用了 Gauge 和 Counter。所以我们重点就来讲解 Gauge 和 Counter 类型。 2022-08-25 15:48:03 这个时 刻，未必恰好有监控数据啊，那这个 Table 中的数据是哪里来的？ 实际上，Prometheus 有个启动参数，--query.lookback-delta=2m 来控制这个行为，如果配 置为 2m，就表示，Prometheus 会查询 2022-08-25 15:46:03 ~ 2022-08-25 15:48:03 这 2 分钟之间的数据，然后返回最新的那个。

Developer Prometheus team member Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih@{fosdem,debian,richih}.org, @TwitchiH OpenMetrics Introduction Quick intro OpenMetrics Outro Prometheus What’s What’s Prometheus? You can’t talk about OpenMetrics without mentioning Prometheus Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih@{fosdem,debian,richih}.org, @TwitchiH OpenMetrics Introduction Quick Quick intro OpenMetrics Outro Prometheus What’s Prometheus? Show of hands: Who has heard of Prometheus? Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih@{fosdem,debian,richih}.org, @TwitchiH

• prometheus • ⽀支持任意维度label • cncf基⾦金金会 metric • 40w+/s的指标采集 • 10k+ 监控⽬目标 • 10+ prometheus节点 现状: • 性能 • ⾼高可⽤用 • 分布式 • 使⽤用成本 问题: ? 性能问题 • 本地ssd prometheus Shard A prometheus Shard Shard B prometheus Shard C targets hash • horizontal sharding
 (实验性质使⽤用) • prometheus 2.0 (tsdb) HA prometheus server1 server2 server3 prometheus IDC HA prometheus server1 server2 server3 server3 prometheus IDC Federation pr s s s pr I pr s s s pr I IDC1 IDC2 prometheus prometheus filter数据 精度降低 建议 降低使⽤用成本 agent prometheus target target target alert_manager 告警平 服务 cache

Endpoint Lifecycle Troubleshooting Monitoring & Metrics Cilium Metrics Hubble Metrics Example Prometheus & Grafana Deployment Metrics Reference Performance & Scalability Tuning Guide CNI Performance running workloads and based on arbitrary label definitions. Metrics export via Prometheus: Key metrics are exported via Prometheus for integration with your existing dashboards. Hubble [https://github.com/cilium/hubble/]: Discovery Network Policy Setting up Support for External Workloads (beta) Operations Running Prometheus & Grafana Istio Getting Started Using Istio The best way to get help if you get stuck is to

Endpoint Lifecycle Troubleshooting Monitoring & Metrics Cilium Metrics Hubble Metrics Example Prometheus & Grafana Deployment Metrics Reference Scalability Limiting Identity-Relevant Labels Scalability running workloads and based on arbitrary label definitions. Metrics export via Prometheus: Key metrics are exported via Prometheus for integration with your existing dashboards. Hubble [https://github.com/cilium/hubble/]: Local Redirect Policy (beta) Operations Networking and security observability with Hubble Running Prometheus & Grafana Istio Getting Started Using Istio Other Orchestrators Cilium with Docker & libnetwork

Endpoint Lifecycle Troubleshooting Monitoring & Metrics Cilium Metrics Hubble Metrics Example Prometheus & Grafana Deployment Metrics Reference Performance & Scalability Tuning Guide CNI Performance the sender and receiver among a lot of other information. Metrics export via Prometheus: Key metrics are exported via Prometheus for integration with your existing dashboards. Hubble [https://github.com/cilium/hubble/]: Discovery Network Policy Setting up Support for External Workloads (beta) Operations Running Prometheus & Grafana Istio Getting Started Using Istio The best way to get help if you get stuck is to

描述 MySQL 提供数据库服务，可配置使用外部数据库服务，例：RDS Ansible CloudExplorer 基于 Ansible 实现自动化操作功能 Prometheus CloudExplorer 基于 Prometheus 实现监控和告警功能 Redis 提供缓存服务 ElasticSearch 提供日志存储服务，可配置使用外部 ES 服务 Keycloak 提供 IDP 服务，用户统一认证中心，可以配置 数据库、Ansible、InfluxDB；  用户认证模块依赖 MYSQL 数据库；  微服务网关依赖管理中心模块与用户认证模块；  其他应用模块都依赖管理中心；  Ansible、Prometheus 等开源组件模块不依赖其他模块； 可通过 yml 文件查看全部依赖，例： 杭州飞致云信息科技有限公司 8 三、部署架构 3.1 高可用部署架构 杭州飞致云信息科技有限公司 9 7600/8080 CloudExplorer IdP 服务端口 idP 服务端口 8080，集群高可用 端口 7600 9090 Prometheus 服务端口 监控使用 9091 Prometheus pushgateway 服务端 口 监控使用 9093 Prometheus alertmanager 服务端口 监控使用 6379 Redis 服务接口 缓存使用 8000/8001 Ansible

running workloads and based on arbitrary label defini�ons. Metrics export via Prometheus: Key metrics are exported via Prometheus for integra�on with your exis�ng dashboards. Integrations Network plugin Flannel (beta) IPVLAN based Networking (beta) Transparent Encryp�on (beta) Operations Running Prometheus & Grafana Istio Ge�ng Started Using Is�o Other Orchestrators Cilium with Docker & libnetwork pods insecure upstream fallthrough in-addr.arpa ip6.arpa } prometheus :9153 proxy . /etc/resolv.conf cache 30 } The contents can look different