Go Big

With over an Exabyte of new data created daily on the Internet, wouldn't it be great if we could utilize Erlang to make sense of it?    

Erlang has great concurrency and built in MapReduce should this be a better fit than Hadoop? 
Unfortunately, today Erlang doesn't scale well enough and it doesn't have the I/O handling capabilities to match what the Hadoop and Grid computing communities can achieve. We'll talk about how they scale to thousands of servers in a cluster and handle gigabytes of I/O per second and and how we can learn from this to  make Erlang scale to match. 

Talk objectives:

Understand how big data systems work today.

Explore the options for improving Erlang and its ecosystem so its becomes the best solution for handling big data.   Side benefits of this would also improve performance in other large Erlang deployments such as some of the Cloud based systems now being built with Erlang

Target audience:

Anybody interested in Erlang performance and scalability or  Big Data  or high performance computing

Slides
Video

About Richard

Richard is a platform architect who specializes in high performance systems, including those used by financial institutions for high frequency trading and huge compute clusters with thousands of nodes used in the Cloud.

Richard discovered Erlang and OTP three years ago and has adopted this as his platform of choice. He has designed and helped build a large and complex Cloud based application using Erlang/OTP and has been exploring the design challenges of getting this to scale to millions of users for the last 12 months.

Richard is a multidisciplinarian with experience across storage, network, operating systems, devOps and systems programming. He has to use all of these when designing a new platform. Over the years, he's programmed in Assembler, Basic, Fortran, Pascal, C, C++, Java, C# and Erlang.

He has worked as VP of High Frequency Engineering at a major bank, Chief Architect at Sun Microsystems, Principle Architect at Microsoft (Azure), as well as a number of interim roles with high tech startups.

Richard is Fellow of STAC Research, a Fellow of the British Computer Society and a Chartered IT Practitioner.

Back to conference page