## Dive into NSQ 舜飞 - 陈冶 ## 消息队列服务 - 面向跨进程/跨服务器通讯的组件 • 异步通信，将可并行化处理的同步操作解耦 ## 使用案例 广告点击数统计 User Click User Click User Click Web Service Stateless MySQL Stat Service Stateless ## 使用案例 广告点击数统计 数据广播，分发给多个服务 • 负载均衡，提高消费者的扩展性 • 消费反馈，确保消息不丢失 • MQ：分布式部署，排除自身单点故障 • MQ：具备横向扩展性，排除性能瓶颈 ## NSQ 是怎么面对这些问题的 ## NSQ概念 • Topic • Channel nsqd ![Image](/uploads/documents/d/c/6/0/dc6055364a3fa6266aef882f0ac0d676/p16_1 同时通过同一个 chan 读取 Channel Consumer Memory Channel chan *Msg Consumer Disk Queue Consumer ## 消费反馈 NSQ Channel InFight Client Msg1 ![Image](/uploads/documents/d/c/6/0/dc6055364a3fa6266aef882f0ac0d676/p21_1

see your Godeps/_workspace/src files "moved" to vendor/. ## NSQ # A realtime distributed messaging platform ## Distributed NSQ promotes distributed and decentralized topologies without single points availability coupled with a reliable message delivery guarantee. See features & guarantees. ## Scalable NSQ scales horizontally, without any centralized brokers. Built-in discovery simplifies the addition of cluster. Supports both pub-sub and load-balanced message delivery. It's fast, too. ## Ops Friendly NSQ is easy to configure and deploy and comes bundled with an admin UI. Binaries have no runtime dependencies

stio控制平面组件原理解析 朱经惠 2018.08.25 ## 关于我 • 朱经惠，ETC车宝平台工程师。 - 喜欢开源，个人开源项目”Jaeger PHP Client”。 - 喜欢研究源码，对NSQ，Jaeger，Istio（控制平面）等go语言开源项目进行过研究。 - 除了代码还喜欢爬山和第二天睡醒后全身酸疼的感觉。 ## 目录 galley Galley provides configuration

• 会触发Redis全量同步（超过repl-backlog-size时） ## 需要替代方案 ## Redis跨机房同步 • 取消Redis的主从同步 - 写主Redis时，同时写一份到NSQ，异步写入其他机房 - 使用SoftLayer的香港云主机作为中转（why?） ## Redis运维 - 内存占用过大时，可以切分为多个实例，减少单个实例的内存占用，减少BgSave和重启时Load数据的时间

add_new_clauses=False) A11(iter, polarity=None) Any(iter, polarity) AtMostOne_NSQ(vals, polarity) AtMostOne_BDD(vals, polarity=None) ExactlyOne_NSQ(vals, polarity) ExactlyOne_BDD(vals, polarity) LB_Preprocess(lits polarity=None, name=None) AtMostOne_NSQ(vals, polarity=None, name=None) AtMostOne_BDD(vals, polarity=None, name=None) AtMostOne(vals, polarity=None, name=None) ExactlyOne_NSQ(vals, polarity=None, name=None) 505 AtMostOne() (Clauses method), 529 AtMostOne_BDD() (Clauses method), 502, 529 AtMostOne_NSQ() (Clauses method), 502, 529 attach_stderr() (in module conda.auxlib.logz), 450 attach_stderr_handler()

add_new_clauses=False) A11(iter, polarity=None) Any(iter, polarity) AtMostOne_NSQ(vals, polarity) AtMostOne_BDD(vals, polarity=None) ExactlyOne_NSQ(vals, polarity) ExactlyOne_BDD(vals, polarity) LB_Preprocess(lits polarity=None, name=None) AtMostOne_NSQ(vals, polarity=None, name=None) AtMostOne_BDD(vals, polarity=None, name=None) AtMostOne(vals, polarity=None, name=None) ExactlyOne_NSQ(vals, polarity=None, name=None) attribute), 493 AtMostOne() (Clauses method), 517 AtMostOne_BDD() (Clauses method), 491, 517 AtMostOne_NSQ() (Clauses method), 491, 517 attach_stderr() (in module conda.auxlib.logz), 440 attach_stderr_handler()

add_new_clauses=False) A11(iter, polarity=None) Any(iter, polarity) AtMostOne_NSQ(vals, polarity) AtMostOne_BDD(vals, polarity=None) ExactlyOne_NSQ(vals, polarity) ExactlyOne_BDD(vals, polarity) LB_Preprocess(lits polarity=None, name=None) AtMostOne_NSQ(vals, polarity=None, name=None) AtMostOne_BDD(vals, polarity=None, name=None) AtMostOne(vals, polarity=None, name=None) ExactlyOne_NSQ(vals, polarity=None, name=None) attribute), 510 AtMostOne() (Clauses method), 532 AtMostOne_BDD() (Clauses method), 507, 532 AtMostOne_NSQ() (Clauses method), 507, 532 attach_stderr() (in module conda.auxlib.logz), 456 attach_stderr_handler()

add_new_clauses=False) A11(iter, polarity=None) Any(iter, polarity) AtMostOne_NSQ(vals, polarity) AtMostOne_BDD(vals, polarity=None) ExactlyOne_NSQ(vals, polarity) ExactlyOne_BDD(vals, polarity) LB_Preprocess(lits polarity=None, name=None) AtMostOne_NSQ(vals, polarity=None, name=None) AtMostOne_BDD(vals, polarity=None, name=None) AtMostOne(vals, polarity=None, name=None) ExactlyOne_NSQ(vals, polarity=None, name=None) attribute), 501 AtMostOne() (Clauses method), 522 AtMostOne_BDD() (Clauses method), 498, 522 AtMostOne_NSQ() (Clauses method), 498, 522 attach_stderr() (in module conda.auxlib.logz), 446 attach_stderr_handler()

add_new_clauses=False) A11(iter, polarity=None) Any(iter, polarity) AtMostOne_NSQ(vals, polarity) AtMostOne_BDD(vals, polarity=None) ExactlyOne_NSQ(vals, polarity) ExactlyOne_BDD(vals, polarity) LB_Preprocess(lits polarity=None, name=None) AtMostOne_NSQ(vals, polarity=None, name=None) AtMostOne_BDD(vals, polarity=None, name=None) AtMostOne(vals, polarity=None, name=None) ExactlyOne_NSQ(vals, polarity=None, name=None) attribute). 492 AtMostOne() (Clauses method), 514 AtMostOne_BDD() (Clauses method), 490, 514 AtMostOne_NSQ() (Clauses method), 490, 514 attach_stderr() (in module conda.auxlib.logz), 439 attach_stderr_handler()

add_new_clauses=False) A11(iter, polarity=None) Any(iter, polarity) AtMostOne_NSQ(vals, polarity) AtMostOne_BDD(vals, polarity=None) ExactlyOne_NSQ(vals, polarity) ExactlyOne_BDD(vals, polarity) LB_Preprocess(lits polarity=None, name=None) AtMostOne_NSQ(vals, polarity=None, name=None) AtMostOne_BDD(vals, polarity=None, name=None) AtMostOne(vals, polarity=None, name=None) ExactlyOne_NSQ(vals, polarity=None, name=None) attribute). 492 AtMostOne() (Clauses method), 514 AtMostOne_BDD() (Clauses method), 490, 514 AtMostOne_NSQ() (Clauses method), 490, 514 attach_stderr() (in module conda.auxlib.logz), 439 attach_stderr_handler()