Consumer error, crash or down is a classic tech problem for applications involves producer and consumer. When guaranteed message delivery is a mandatory at consumer side, it is essential to have a general architecture or framework in place thus free the consumer of complicated logic such as error recovery or message re-delivery. 

In this talk, we present a distributed framework which guarantees message delivery for producer/consumer applications. Producer and consumer can be distributed across network, no shared memory or state is needed, which fits Erlang process

model very well. This framework suites well for applications/tasks that can be abstracted using producer/consumer model or chained

producers/consumers.

A general guaranteed message delivery framework is built around open source no sql db redis. The framework has a redelivery mechanism, which redelivers message at configurable interval. Consumer feedback is essential in the control loop of the framework, which is used to remove message from retry or schedule retry at alternative pace. With the redeliver mechanism of the framework, consumer code does not need to worry about how to recover from a crash or downstream error while processing message/event. It can simply ignore or log an error and the system will put the same message/event in the consumer queue for redelivery at certain time later.

With building blocks, an Erlang lib for producer/consumer, we believe the system can benefit the community and developers for easily building system or applications that provides reliable message delivery.

Talk objectives: This distributed, reliable message delivery system has been working great for us. We'd like to share design ideas with the community and hopefully open source the whole system.

Target audience: Developers who build application/system that involves producer and consumer. And those who are interested in reliable message delivery.