IDL breaking check IDL Debug IDL Mock ... API Gateway Framework（Ego） Auth Interceptor Metrics IDL Docs Kubernetes Redis CMP DB CMP Log Interceptor Err Interceptor Metric Interceptor Builtin Interceptors • Authentication Interceptor • Logging Interceptor • Tracing Interceptor • Metrics Interceptor • Debug Interceptor • Errors Interceptor • ... Framework - Builtin Interceptors framework logger Observability - Metrics • Based on Kubernetes/Prometheus/Grafana • Builtin framework and Go runtime collectors，Export APIs for user to add their own metrics • Declare Prometheus annotations

the realization �27 1. Selection of performance metrics 2. Finding how to control rapidly and continuously Performance metrics Performance metrics �29 • Independent resource type which task use blocking tasks into as CPU bound as possible by switching tasks continuously. Performance metrics �30 • The metrics upper limit is calculated on running. 1. Set the target value high Performance 0 20 10 Step 1 2 3 4 5 6 7 8 9 10 Goroutines Performance Performance target Performance metrics �31 • The metrics upper limit is calculated on running. 1. Set the target value high 2. gradually adjust

state ModuleBlockState // module state metrics metrics.Metrics } func NewModuleBlock() *ModuleBlock { m := new(ModuleBlock) m.metrics.Init(&m.state, ModBlock, 0) ... } // register

● Check Hygiene ○ Service Metadata ○ Image Version ● Quality metrics ○ API Test Coverage ○ Flakiness ● Security metrics ○ Security Test Coverage Production Deployment Overview 03. Detailed 04. What can we do to help? When things are broken…. You want to know what happened? Logging Metrics Tracing Structured Logging 04. Improve observability with custom logging library with our ELK

Go 应用监控系统演进 张平 AfterShip 高级 SRE 关于 AfterShip 拥抱云原生和开源系统 目 录 监控架构概览 01 如何监控 Go 应用？ 02 Metrics 系统架构演进 03 Why VictoriaMetrics so good？ 04 总结与展望 05 监控架构概览 第一部分 监控系统架构概览 -- 数据源 监控系统架构概览 -- infra_http_request_total ● 业务指标 ○ 总 Tracking 查询量 ○ Tracking 创建速率 ○ 某个 ENT 客户的 Tracking 查询失败率 Metrics 系统架构演进 第三部分 2018-2020 2K+ 40K 1Mil+ 2020 年指标数据 业务指标数量 每秒写入数据点 Active Time Series 2018-2020

个别不在意延迟的选择第三⽅库优 化 压测⼿段 公司内部压测平台 全链路压测 阻塞导致⾼延迟 在后端系统开发中，锁瓶颈是较常⻅的问题，⽐如⽂件锁 阻塞导致⾼延迟 还有⼀些公司的 metrics 系统设计，本机上会有 udp 通信 阻塞导致⾼延迟 锁瓶颈的⼀般优化⼿段： • 缩⼩临界区：只锁必须锁的对象，临界区内尽量不放慢操作，如 syscall • 降低锁粒度：全局锁 -> 对象锁，全局锁 connection level lock • Connection level lock -> request level lock • 同步改异步 • ⽇志场景：同步⽇志 -> 异步⽇志 • Metrics 上报场景：select -> select+default • 个别场景使⽤双 buffer 完全消灭阻塞 Continuous Profiling 第三部分 难以发现的偶发问题 压测是⼀个蹲点⼉⾏为

none (默认值 none) 2020 © Changkun Ou · Go 夜读 · 对 Go 程序进行可靠的性能测试 benchstat 的原理：异常值消除+假设检验 11 type Metrics struct { Unit string // 性能测试的名称 Values []float64 // 某个性能测试的度量值 RValues []float64 RValues 的最小值 Mean float64 // RValues 的平均值 Max float64 // RValues 的最大值 } func (m *Metrics) computeStats() { values := stats.Sample{Xs: m.Values} q1, q3 := values.Percentile(0.25), values

including total throughput and throughput per API Pr Promet ometheus heus To collect Casdoor's runtime metrics, such as API Throughput, API Latency, CPU Usage, Memory Usage, and more, you need to configure your following information in Prometheus: global: scrape_interval: 10s # The time interval for fetching metrics scrape_configs: - job_name: 'prometheus' static_configs: - targets: ['localhost:9090'] - job_name: static_configs: - targets: ['localhost:8000'] # Back-end address of Casdoor deployment metrics_path: '/api/metrics' # Path for collecting indicators Int Internationalization ernationalization Casdoor

微服务系统监控的挑战 01 可观测性技术理论 02 Erda 服务观测平台技术内核分析 03 Erda 服务观测平台功能概览 04 可观测性三大支柱 （一）Tracing 模型 （二）Metrics 时序模型 Writes are vertical，reads are horizontal （三）数据关联 目 录 微服务系统监控的挑战 01 可观测性技术理论 02 Erda 服务观测平台技术内核分析

OpenCensus (originally developed by Google)Telemetry signals - Logs (additonal info about behaviour) - Metrics (measuring) - TracesTrace A trace represents the execution path of a request across multiple services