using a client-side load-balancing library + Headless Services Headless services are to us what ClusterIP services are to common people! However, our KubeDNS was not happy at all with the SRV requests headless services worse. Conclusion: We stopped using headless services and gradually migrated to ClusterIP services ## Adopting Istio ## The hell of migrating hundreds of services - Services are immutable for each service migration, we need to: Write the ClusterIP service equivalent ☐ Make sure Istio-enabled callers update their config with the ClusterIP service ☐ Keep a double standard during migration

echo-b-host-67446447f7-chvsp 1/1 Running 0 4m50s host-to-b-multi-node-clusterip-78f9869d75-l8cf8 1/1 Running 0 4m50s host-to-b-multi-node-headless-798949bd5f-vvfff pod-to-b-intra-node-845f955cdc-5nfrt 1/1 Running 0 4m49s pod-to-b-multi-node-clusterip-666594b445-bsn4j 1/1 Running 0 4m49s pod-to-b-multi-node-headless-746f84dff5-prk4w echo-b-host-67446447f7-chvsp 1/1 Running 0 4m50s host-to-b-multi-node-clusterip-78f9869d75-l8cf8 1/1 Running 0 4m50s host-to-b-multi-node-headless-798949bd5f-vvfff

2. ClusterIP Service 场景下的服务发现 对于这类服务，在创建K8s的Service对象后， CoreDNS会新建一条ServiceName->ClusterIP的记录，一般情况下，ClusterIP不会变化。 客户端进行访问时，通常会指定 ServiceName 来进行访问。通过 CoreDNS 将 ServiceName 解析为 Service对应的 ClusterIP。当应用程序通过 ClusterIP。当应用程序通过 DNS 拿到 ClusterIP 后，直接发起对 ClusterIP 的连接，由 Kube-Proxy 通过 iptables/IPVS 发送至一个真实的 Pod IP。 YAML apiVersion: v1 kind: Service metadata: name: my-app spec: type: ClusterIP clusterIP: 10.96.0.101 从 Kitex Kitex 客户端的角度来看，服务发现（DNS解析）获取到的始终都是 ClusterIP，实际访问到的实例的 ip在连接建立前是无从得知的。在这种情况下，Kitex内的服务发现结果刷新是无意义的。 这样存在以下两个问题： 其一，无法感知具体实例。没有实例信息，框架侧无法进行负载均衡；Kube-proxy的负载均衡只在建立 连接时生效，在连接多路复用场景下，可能导致流量倾斜至单实例。 其二，实例级别熔断失

echo-b-host-6b7fc94b7c-xtsff 1/1 Running 0 66s host-to-b-multi-node-clusterip-85476cd779-bpg4b 1/1 Running 0 66s host-to-b-multi-node-headless-dc6c44cb5-8jdz8 pod-to-b-intra-node-nodeport-9b487cf89-6ptrt 1/1 Running 0 65s pod-to-b-multi-node-clusterip-7db5dfdcf7-jkjpw 1/1 Running 0 66s pod-to-b-multi-node-headless-7d44b85d69-mtscc echo-b-host-6b7fc94b7c-xtsff 1/1 Running 0 66s host-to-b-multi-node-clusterip-85476cd779-bpg4b 1/1 Running 0 66s host-to-b-multi-node-headless-dc6c44cb5-8jdz8

userservice.default.svc.cluster.local -> 10.254.162.44 1. 服务部署 - ServiceType=ClusterIP - ServiceName=userservice - ClusterIP=10.254.162.44 - Domain=userservice.default.svc.cluster.local ## I stio options中的 original dest 获取ClusterIP 根据ClusterIP从 VirtualHost 中匹配Rule规则，得到转发目标地址 Sidecar 5. iptables拦截到10.254.162.44的请求 转发到localhost，打到Sidecar 在TCP options将ClusterIP保存为original dest 7. 请求转发给对端 cluster.local -> 10.254.162.44 Pod IP=192.168.1.103 1. 服务部署 - ServiceType=ClusterIP - ServiceName=userservice - ClusterIP=10.254.162.44 - Domain=userservice.default.svc.cluster.local ## SOFAMesh 的DNS寻址

echo-b-host-f86bd784d-wnh4v 1/1 Running 0 68s host-to-b-multi-node-clusterip-585db65b4d-x74nz 1/1 Running 0 68s host-to-b-multi-node-headless-77c64bc7d8-kgf8p pod-to-b-intra-node-nodeport-8484fb6d89-bwj8q 1/1 Running 0 68s pod-to-b-multi-node-clusterip-f7655dbc8-h5bwk 1/1 Running 0 68s pod-to-b-multi-node-headless-5fd98b9648-5bjj8 echo-b-host-f86bd784d-wnh4v 1/1 Running 0 68s host-to-b-multi-node-clusterip-585db65b4d-x74nz 1/1 Running 0 68s host-to-b-multi-node-headless-77c64bc7d8-kgf8p

spec: type: ClusterIP ports: - port: 9999 targetPort: 9999 protocol: TCP name: http selector: {{- include "influxdb.selectorLabels". | nindent 4 }} spec: type: ClusterIP sessionAffinity: sessionAffinity: None clusterIP: None ports: - port: 9999 targetPort: 9999 protocol: TCP name: http selector: {{-include "influxdb.selectorLabels".|nindent4}} vi influxdb2/templates/statefulset # - ALL # readOnlyRootFilesystem:true # runAsNonRoot:true # runAsUser:1000 service: type: ClusterIP port:8888 ingress: enabled:false annotations:{} # kubernetes.io/ingress.class:nginx # kubernetes

echo-b-host-6b7fc94b7c-xtsff 1/1 Running 0 66s host-to-b-multi-node-clusterip-85476cd779-bpg4b 1/1 Running 0 66s host-to-b-multi-node-headless-dc6c44cb5-8jdz8 pod-to-b-intra-node-nodeport-9b487cf89-6ptrt 1/1 Running 0 65s pod-to-b-multi-node-clusterip-7db5dfdcf7-jkjpw 1/1 Running 0 66s pod-to-b-multi-node-headless-7d44b85d69-mtscc echo-b-host-6b7fc94b7c-xtsff 1/1 Running 0 66s host-to-b-multi-node-clusterip-85476cd779-bpg4b 1/1 Running 0 66s host-to-b-multi-node-headless-dc6c44cb5-8jdz8

58. oc create service clusterip 创建 ClusterIP 服务。 ## 用法示例 # Create a new ClusterIP service named my-cs oc create service clusterip my-cs --tcp=5678:8080 # Create a new ClusterIP service named my-cs (in (in headless mode) oc create service clusterip my-cs --clusterip="None" ###### 2.5.1.59. oc create service externalname 创建 ExternalName 服务。 ## 用法示例 # Create a new ExternalName service named services # Observe changes to services, including the clusterIP and invoke a script for each oc observe services --template '{.spec.clusterIP}' -- register_dns.sh # Observe changes to services

Running066shost-to-b-multi-node-clusterip-85476cd779-bpg4b1/1Running066sRunning065spod-to-b-multi-node-clusterip-7db5dfdcf7-jkjpw1/1Running066sRunning066shost-to-b-multi-node-clusterip-85476cd779-bpg4b1/1Running066s