Scott Lystig Fritchie
Decade-long Erlang addict
Basho Technologies
Scott Lystig Fritchie met his first UNIX system in 1986 and has almost never met one since that he didn't like. A career detour as a UNIX systems administrator got him neck-deep in messaging systems, e-mail and Usenet News. He rediscovered full-time programming while at Sendmail, Inc., where a colleague introduced him to Erlang in 2000. His world hasn't been the same since then.
In addition to hacking Erlang code and occasionally the Erlang virtual machine, he's had papers published by USENIX, the Erlang User Conference, and the ACM.
In addition to hacking Erlang code and occasionally the Erlang virtual machine, he's had papers published by USENIX, the Erlang User Conference, and the ACM.
Scott Lystig Fritchie is Giving the Following Talks
Webmail for millions, powered by Erlang
How do you design and implement a custom, complex web-based email system that must be robust, scalable across dozens of machines, and support two million users at the start? The small size of your entire staff is a handicap. Everything must be tested to the rigorous standards of telecom giant in Asia which overlooks very, very few details.
I'll describe one way to do it, using Erlang at its heart for:
* Communicating with the Web browser client
* Speaking SMTP, POP, and IMAP to the rest of the email world
* Interfacing with a proprietary, external text index/search and user authentication services
* Gluing C++ components for speed-critical and legacy feature support
* Using a dynamically scalable key-value DB to store everything.
All email, user profile, and vCard data is stored in a
distributed key-value database that supports dynamic online resizing as the service grows while keeping email stored safely and consistently.
We make heavy use of protocol contracts (using UBF) to enforce over-the-wire sanity of all intra-system communication and with the browser. It's tested extensively with EUnit, QuickCheck, and Python scripts. I'll give details of what has worked well and what we should have done differently through each stage of design, implementation,
testing, and deployment.
I'll describe one way to do it, using Erlang at its heart for:
* Communicating with the Web browser client
* Speaking SMTP, POP, and IMAP to the rest of the email world
* Interfacing with a proprietary, external text index/search and user authentication services
* Gluing C++ components for speed-critical and legacy feature support
* Using a dynamically scalable key-value DB to store everything.
All email, user profile, and vCard data is stored in a
distributed key-value database that supports dynamic online resizing as the service grows while keeping email stored safely and consistently.
We make heavy use of protocol contracts (using UBF) to enforce over-the-wire sanity of all intra-system communication and with the browser. It's tested extensively with EUnit, QuickCheck, and Python scripts. I'll give details of what has worked well and what we should have done differently through each stage of design, implementation,
testing, and deployment.
