. 4 4.2 Message Priority ............................................................................................................................................... 4 4.3 Message Order ..... 5 4.4 Message Filter ................................................................................................................................................... 5 4.5 Message Persistence ...................................................................................... 5 4.6 Message Reliablity .....................................................................................

进程自动退出排查经验分享 78 1.8 RocketMQ 主题扩分片后遇到的坑 82 1.9 RocketMQ 消息发送 system busy、broker busy 原因分析与解决方案坑 91 1.10 再谈 RocketMQ broker busy 104 1.11 从年末生产故障解锁 RocketMQ 集群部署的最佳实践 108 1.12 RocketMQ 一行代码造成大量消息丢失 Nameserver Nameserver 集群，topic 的路由注册中心，为客户端根据 Topic 提供路由服务，从 而引导客户端向 Broker 发送消息。Nameserver 之间的节点不通信。路由信息在 Nameserver 集群中数据一致性采取的最终一致性。 2. Broker 消息存储服务器，分为两种角色：Master 与 Slave，上图中呈现的就是 2 主 2 从的部 署架构，在 RocketMQ RocketMQ 中，主服务承担读写操作，从服务器作为一个备份，当主服务器存 在压力时，从服务器可以承担读服务（消息消费）。所有 Broker，包含 Slave 服务器每隔 30s 会向 Nameserver 发送心跳包，心跳包中会包含存在在 Broker 上所有的 topic 的路 由信息。 本文来自『中间件兴趣圈』公众号，仅作技术交流，未授权任何商业行为。 11 > 1.1 RocketMQ

collector. It automatically reacts when a log occurs, and sends the log details (level, logger name, message, etc) to the appenders. The decanter-collector-log feature installs the log collector: karaf@root()> osgi:DecanterLogCollectorAppender, osgi:VmLogAppender 1.2.2. CXF LOGGING FEATURE INTEGRATION The CXF message logging nicely integrates with Decanter. Simply add the org.apache.cxf.ext.logging.LoggingFeature attributes. The message logging can be switched on/off per service using the org.ops4j.pax.logging.cfg. When using with Decanter make sure you enable the log collector to actually process the message logs. Apache

collector. It automatically reacts when a log occurs, and send the log details (level, logger name, message, etc) to the appenders. The decanter-collector-log feature installs the log collector: karaf@root()> DecanterLogCollectorAppender appender: 1.2.2. CXF Logging feature integration The CXF message logging (http://cxf.apache.org/docs/message-logging.html) nicely integrates with Decanter. Simply add the org.apache.cxf attributes. The message logging can be switched on/off per service using the org.ops4j.pax.logging.cfg. When using with Decanter make sure you enable the log collector to actually process the message logs.

symbolic name --context, -c Use the given bundle context (defaults to 0) --help Display this help message -t Specifies the bundle threshold; bundles with a start-level less than this value will not get output The example that is installed is using Camel to start a timer every 2 seconds and output a message in the log. The previous commands downloaded the Camel features descriptor and installed the example org/schema/blueprint"> Message at ${date:now:yyyy-MM-dd HH:mm:ss}

sources Files, e.g. transaction logs Sockets IoT devices and sensors Databases and KV stores Message queues and brokers Where do stream processors read data from? 2 Challenges • can be distributed on the network • Failure handling: application needs to be aware of message loss, producers and consumers always online 5 Message queues • Asynchronous point-to-point communication • Lightweight buffer guarantees • Each message is processed only once, by a single consumer • Event retrieval is not defined by content / structure but its order • FIFO, priority producer consumer queue 6 Message brokers Message

architecture� 典型应⽤用场景 ⾦金金融交易易 电⼦子商务 智能制造 分布式事务 异步解耦 IoT/IIoT 决策分析 实时计算 概念模型 Broker A Producer A Topic A Broker B Topic B ConsumerGroupA ConsumerGroupB Consumer 1 Consumer 2 Producer B Consumer Consumer 1 Consumer 2 MessageQueue1 MessageQueue2 MessageQueue1 MessageQueue2 部署模型 组件细节 Broker Netty RemotingServer Netty Remoting Client NameServer All Info In Memory except KV config info RocketMQ 存储设计 存储⽂文件 - - - - - 1G - - - - - 6M - - - - - 400M 配置和状态相关 消息存储和检索相关 存储设计 producer message(topic, queueId, tags, keys, msgBody) commitlog 8 bytes | 4 bytes | 8 bytes commitLogOffset

designed for B2B Trading and other scenarios MetaQ v1.0 Designed for ordered message, especially for message accumulation Aliware MQ v1.0 Notify v3.0 MetaQ v2.0 v3.0 RocketMQ v3.0 and Non-Ordered message  Broadcast message  Scheduled message  Transactional message  Batched message  One-way message  Traceable message  Retroactive message  Message Filtering  Massive e G r o u p Console Scheduled Message © 2 0 1 7 A l i b a b a M i d d l e w a r e G r o u p Transactional Message – Eventually Consistent © 2 0 1 7 A l i b a b a M i

吞吐、高可靠、低延迟方面的要求。 目前RocketMQ主要由NameServer、Broker、Producer以及Consumer四部分构成，如下图所示。 原文链接：Apache RocketMQ 介绍 NameServer以轻量级的方式提供服务发现和路由功能，每个NameServer存有全量的路由信息，提 对等的读写服务，支持快速扩缩容。 Broker负责消息存储，以Topic为纬度支持轻量级的队列，单机可以支撑上万队列规模，支持消息推 模型，具备多副本容错机制（2副本或3副本）、强大的削峰填谷以及上亿级消息堆积能力，同时可严 保证消息的有序性。除此之外，Broker还提供了同城异地容灾能力，丰富的Metrics统计以及告警机 。这些都是传统消息系统无法比拟的。 Producer由用户进行分布式部署，消息由Producer通过多种负载均衡模式发送到Broker集群，发送 延时，支持快速失败。 Consumer也由用户部署，支持PUSH和PULL两种消费 RocketMQ单机最高支持5万个队列，且负载不会发生明显递增。 ● RocketMQ支持消费失败重试，这个特性非常适合运用在充值方面的应用。 ● RocketMQ支持严格的意义上的消息顺序。即时在一台Broker宕机后，消息发送会失败，但是不会 序。 ● RocketMQ支持定时消息 ● RocketMQ支持分布式事务消息 ● RocketMQ支持根据消息标识或内容查询。 ● RocketMQ支

Scalers for RocketMQ with KEDA Install tools/ softwares - JDK, Maven, Git, 4g+ free disk for Broker server, terminal, Apache RocketMQ - Helm, Kubernetes, ​kubectl​,​Docker, KEDA, Golang Basic deployment, we could validate the deployment scales, checking the pods and start process queue message, if message length increase, more pods will be pro-actively added - We can also get the number of messages Benchmark the ScaledObject will be deployed when there is change in the scaling as the number of message increases pollingInterval: 30 # Optional. Default: 30 seconds cooldownPeriod: 300 # Optional