������������������� StatefulSetPlus Operator Ø Keep share memory during Pod upgrade Ø Upgrade jitter (a few ms) for keep-alive services Flexible and dynamic resource management Dynamic Scheduler is

Use envelope encryption scheme • DEK & KEK Motivation: K8s Secrets Protection • Performance & latency • Network • Security • DEK in the clear in memory • Secret in the clear in memory • kubeconfig • Encrypted memory • SW/HW attacks prevented TEE-based KMS Plugin [1] • Address performance & latency concerns • Reduce / minimize remote KMS interactions w/o compromising security • Address security force update • Liveness probe • Monitoring • Integration w/ Prometheus • Metrics including • latency of en/decryption • failure times of en/decryption • KMS health check • Ops tooling • kms-plugin-tools

Online service Batch jobs Category Online shopping web apps, payment service MR, spark, flink Latency Sensitive Insensitive Priority high low Traffic pattern Peak at day time Peak at night time Fault

强大的工具箱 ● PodChaos: kill / fail / ... ● NetworkChaos: delay / lose / dup / partition / … ● IOChaos: latency / fault / … ● TimeChaos: clock skew ● KernelChaos: kernel fault injection ● StressChaos: burn

measurement Test topology Test result Service type Short connection cps Short connection P99 latency Long connection pps ClusterIP +40% -31% not available NodePort +64% -47% +22% Test result •

function and process — to provide the best opportunity for service recipient success. — Gartner Latency SLI Availability QPS Correctness SLO …… Punishment SLA SLI defines an indicator, which can