A Layered Architecture: Robustness and Efficiency
| The redesign of our technology stack into robust and cleanly separated layers. The importance of efficient and strict communication between them. Talk objectives: Giving an understanding of the problems when reimplementing an existing architecture. Sharing the technologies we've used and some insight on the roadblocks we've encountered. Target audience: Architects, CTOs, backend developers. |
All your Cores are Belong to us: Turbo-Charging for the Many-Core World
| Erlang provides an excellent framework for programming concurrent systems on multi and manycore CPUs. In this talk we will discuss the work we have done at Concurix in enhancing Erlang and the underlying operating system to take advantage of manycore systems. We will cover both real world benchmark results as well as detail some of the underlying technology improvements such as parametric memory configuration, automatic memoization, reducing locks, and so forth. Talk objectives: To disseminate our learnings from making Erlang run faster in many-core machines, and share our development results with the Erlang community. Target audience: Erlang developers and general web application developers interested in switching to Erlang. |
Amateur Radio Propagation Analysis with Erlang/OTP and Riak
| Automated signal analysis in multiple reception locations enables a large-scale data analysis opportunity for radio propagation. The accumulated datasets result in millions of reception reports, which is suitable for applying "big data" solutions. In this presentation, Kenji describes a problem solution approach with Erlang/OTP and Riak to store and analyze the reception reports from two different amateur radio shortwave reception networks called reversebeacon.net and WSPRnet, and shows how to achieve a good computational performance. Talk objectives: Showing how a KVS-based approach is different from the traditional SQL-based database approach, and how Erlang/OTP and Riak can effectively utilize the distributed and multi-core system environment. Target audience: Entry users of database applications on Erlang/OTP, including not only those of Riak, but also of the other "NoSQL" database implementations. |
Analyzing Erlang with Big Data Techniques
| Applying big data ideas, methods, and implementations, we at Concurix have identified opportunities for optimizing the performance of Erlang applications. This talk will present details of the data and analyses of Erlang VM and Erlang applications. The performance gain obtained on Erlang applications using the big data approaches will be reported. As for big data in general, we will share some of our experiences in data tools. Talk objectives: To showcase the power of big data and analytics in optimizing for scalability and performance wins. Target audience: Erlang Developers. |
Beating the No-Win Scenario: Testing a Distributed System
| Testing distributed systems is hard. Testing is hard, but we've been given tools like EUnit and common_test to make it easier. Where is this toolkit for distributed systems? I'm not going to claim that I have all the answers or even that I have a framework you can use to solve your problem. What I do have is an idea and a solution to MY problem, which is ""How do we test Riak?" Come with me on an adventure of design decisions, tradeoffs and deep dark corners of Erlang. We'll go from group leaders to lager and port commands to cookies. By journey's end you'll have an understand of how we're able to test riak, how you may be able to build off what we've done to test your distributed system, and hopefully ideas on how we might, together, make riak_test generic enough to be that distributed testing tool we've been looking for. Talk objectives: Riak_test is not a conventional Erlang application. It doesn't use supervisors and is at its core, an escript. This talk hopes to bring attention to features of Erlang and how they can be used to test an Erlang application. This talk hopes to explore corner cases of Erlang features, that are most likely not being used as intended to test an erlang application. Some of the more interesting things I'll show off are: * How we simulate a network partition * How we are able to test console interaction * Using group_leader to capture all kinds of output * Using a custom lager backend for keeping logs in memory * Using module attributes to check for test prerequisites I also hope to drum up more interest in riak_test from outside of Basho, in an effort to make it a more generic framework for testing distributed systems. Target audience: I think even people with a lot of Erlang experience will be able to get something out of this talk, even if that is just a new perspective on old material. New users will get to see some of these features explored, giving them something else to think about besides list comprehensions, gen_servers and supervision trees. Developers new and old alike will be exposed to the issues faced in testing a distributed system, and will hopefully consider those in the future as they write code that will, one day, need testing. |
| Noah, Mario and James will deliver a three-part talk. Part 1: Noah Gift will talk about how he believes in Erlang and how that factors into hiring, architecture and business dollars and cents at a game company. Part 2: Mario Izquierdo will take you for a deep dive into a RabbitMQ integration into a Rails game architecture, DIO, and how Erlang "made it scale". Part 3: James Mayfield will go into the evolution of an AI game written in Erlang as it went from prototype to "shipped". Talk objectives: To explain how Erlang fits into game and web development in games. Target audience: Erlang developers, web developers and anyone interested in games. |
| The goal of Disco has been to be a simple and usable implementation of MapReduce. To keep things simple, this MapReduce aspect has been hard-coded into Disco, both in the Erlang job scheduler, as well as in the Python library. To fix various issues in the implementation, we decided to take a cold hard look at the dataflow in Disco's version of MapReduce. We came up with a generalization that should be more flexible and hence also more useful than plain old MapReduce. We call this the Pipeline model, and we hope to use this in the next major release of Disco. This will implement the old MapReduce model in terms of a more general programmable pipeline, and also expose the pipeline to users wishing to take advantage of the optimization opportunities it offers. If time permits, we will also discuss other aspects of the Disco roadmap, and the future of the Disco project. Talk objectives: Introduce features in the upcoming release of Disco, and talk about their design and motivation, and how Disco users could take advantage of them. Target audience: Big data hackers and Disco users. |
Bottleneck Whack-A-Mole at Scale
| When an Erlang app starts falling over in production it can seem like a daunting task getting it back up and stabilized again. We will discuss a number of methods, both reactive and preventative, for bulletproofing Erlang apps against concurrency bottlenecks, VM quirks and the plethora of ways your code can come back to haunt you. Talk objectives: Teach the audience how to bulletproof their Erlang code against scaling pains. Target audience: Folks who write and support production code. |
Browser Frontend + Erlang Backend-> Webapp Anywhere
| This talk is mainly about how we utilize Erlang to build a unified platform that can act as a solid, concurrent backend for HTML5 webapps. Qiyun Webapp platform provides the whole tool chain and a unified platform for designing, developing, deploying and hosting HTML5 webapps. It includes project management, online editor, WYSIWYGE IDE,version control, database design and operation tools, security control, all integrated.Users can write their backend logic in Javascript which is parsed and compiled to run in Erlang VM. More details can be found in http://qyapp.com/qiyun/website/quickstart-en.html. Erlang is the infrastructure and the power horse that makes all these possible. In this talk I will give technical details about how all these features are implemented in Erlang and why we think Erlang is a natural fit in building cloud platform for webapps. With browser and Javascript, HTML5,CSS3 as UI frontend and Erlang as backend, we dare say that this mode is the future of any apps. Talk objectives: First we like to demostrate how Erlang helps in building this platform. How its uniqueness and technical advantages make this platform possible. We like to give our example to tell everybody that Erlang is a natural fit in this kind of task and it should have more influence in future webapp trends. Second we like to show our technologies to draw more attention. Also we like to share our view of how webapp or any apps should be developed in future. Lastly, we like to know more people that are interested in Erlang and find its excellence. Target audience: People who are interested in seeing Erlang used in the cloud. People who are interested in building webapps in a modular, consistent and unified way where they can write UI and backend logic in a single language--Javascript. People who like new technical innovations which combines the backend and front end in a seamless way, utilizing the power of Erlang to provide a massive concurrent run time for webapps running anywhere--desktop, ipad, mobiles. |
| Riak CS is an Erlang application that exposes the S3 API on top of Riak: an eventually consistent fault-tolerant distributed key-value store. Riak CS design faced many challenges: supporting large files, distributed garbage collection and quick time to market. We'll dive into implementation details and lessons applicable to anyone building distributed systems, such as the power of immutability in large-scale system design. Time will also be given to less obviously exciting issues like versioning of persisted formats and rolling upgrades. Talk objectives: To relay lessons and tips from building Riak CS with Erlang and Riak. Target audience: Erlang developers interested in learning about a large-scale distributed system. |
Building a Messaging and Service Protocol
| This talk presents the architecture behind a new backend for a game. The backend exhibits stateful design, modular components, and supports many concurrent users. Communication is built atop Cowboy websockets and it is pluggable. At its core, the backend is a chat server. However, entities on the server can be users or services. Services are essentially chat bots with programmable capabilities. API requests to the server resemble chat messages to services so the client is easy to program. Adding new functionality is simply a matter of modifying an existing service or spinning up a new one with a rebar template. The services themselves modify the user's state in memory as well as on the database so the entire flow requires very few database reads. Talk objectives: This talk aims to showcase Erlang usage in the mobile games department. It is a case study talk at its core and presents a novel architecture that is easy to understand and even easier to develop for. Target audience: The talk appeals to individuals early on in the development process of a new Erlang project or curious about actual industry usage of Erlang. |
Chicago Boss: A Web Framework Built for Comfort (and Speed)
| We all know that Erlang is great for building complicated distributed systems, but did you know that building a website in Erlang is as easy as using Ruby on Rails? In this talk I'll introduce you to Chicago Boss, the web framework that takes care of all the hard parts of Erlang/OTP for you, letting the developer focus on building a great website. I'll explain why Erlang is an excellent choice for delivering web content, give a high-level overview of the Chicago Boss architecture, and explicate the philosophy behind some of CB's controversial design decisions. In the process we'll cover CB's high-performance template system, its custom compiler chain and language-integrated query syntax, its unique approaches to e-mail and i18n, and everything in between. Some practical code examples will be included at the end. All levels of expertise are welcome. Talk objectives: The goal of the talk is to help attendees to appreciate the strength of Erlang as a web development platform, gain a high-level understanding of Chicago Boss internals, and get inspired to build their next website with Chicago Boss! Target audience: This talk will appeal to Erlang developers looking to expand their toolbox, web developers who are new to Erlang, and anyone who thinks a person must be certifiably insane to want to build a website in Erlang. |
Code Janitor: Nobody's Dream, Everyone's Job
| Most software engineers and programmers like to spend their time implementing new features and exploring new ideas. Sadly for them, 70% to 80% of a product's life cycles are spent in maintenance mode. Why does it take so much time? Generally, of most the maintenance time is spent in code, trying to understand how the system works, building a mental model of it, so that we can later change the program safely. Erlang and OTP make it possible to reduce the effort required for this maintenance. This talk will explore and suggests explanations as to why software maintenance is made better with the Erlang/OTP platform. Talk objectives: This presentation seeks to identify the nature of software maintenance, then to establish Erlang best practices to make sure the daily maintenance tasks do not become a depressing house of horrors, but instead remain as motivating as the writing of new code. Target audience: Beginners to intermediary Erlang programmers. It's a relatively high-level presentation, and a general understanding of the Erlang/OTP principles is recommended. |
Distributed Producer and Consumer Framework with Guaranteed Message Delivery
| Consumer error, crash or down is a classic tech problem for applications involves producer and consumer. When guaranteed message delivery is a mandatory at consumer side, it is essential to have a general architecture or framework in place thus free the consumer of complicated logic such as error recovery or message re-delivery. In this talk, we present a distributed framework which guarantees message delivery for producer/consumer applications. Producer and consumer can be distributed across network, no shared memory or state is needed, which fits Erlang process model very well. This framework suites well for applications/tasks that can be abstracted using producer/consumer model or chained producers/consumers. A general guaranteed message delivery framework is built around open source no sql db redis. The framework has a redelivery mechanism, which redelivers message at configurable interval. Consumer feedback is essential in the control loop of the framework, which is used to remove message from retry or schedule retry at alternative pace. With the redeliver mechanism of the framework, consumer code does not need to worry about how to recover from a crash or downstream error while processing message/event. It can simply ignore or log an error and the system will put the same message/event in the consumer queue for redelivery at certain time later. With building blocks, an Erlang lib for producer/consumer, we believe the system can benefit the community and developers for easily building system or applications that provides reliable message delivery. Talk objectives: This distributed, reliable message delivery system has been working great for us. We'd like to share design ideas with the community and hopefully open source the whole system. Target audience: Developers who build application/system that involves producer and consumer. And those who are interested in reliable message delivery. |
Erlang Engine Tuning: Part 1 - Know your Engine
| In this talk we will take a look under the hood of the Erlang RunTime System (ERTS). We will look at compiler options to generate readable machine code, at how a process is represented in memory, at the BEAM instruction set and details about how memory is handled. Talk objectives: After the talk you should know how to generate and read BEAM code. You should understand how Erlang data structures are laid out in memory and how the garbage collector work. With this knowledge you should be able to tune the VMs memory handling. Target audience: Dev Ops running applications on Erlang, and Erlang developers who want to get a better understanding of how their programs execute. |
|
Erlang and OpenFlow: A Match Made in the Cloud!
| How our view of networking must evolve to address the multi-core crisis! I'll be describing LINC: an Open Source OpenFlow switching platform developed in Erlang, intended for use with the OTP. I will also introduce OpenFlow, the Open Network Foundation, and the community around LINC called FlowForwarding.org. Talk objectives: Spark debate and interest in solving networking challenges in ultra-scale computing platforms like Erlang/OTP and Scala/AKKA. Target audience: The Erlang/OTP community and emerging ultra-scale computing enthusiasts. How our view of networking must evolve to address the multi-core crisis! I'll be describing LINC: an Open Source OpenFlow switching platform developed in Erlang, intended for use with the OTP. I will also introduce OpenFlow, the Open Network Foundation, and the community around LINC called FlowForwarding.org. Talk objectives: Spark debate and interest in solving networking challenges in ultra-scale computing platforms like Erlang/OTP and Scala/AKKA. Target audience: The Erlang/OTP community and emerging ultra-scale computing enthusiasts. |
Erlang does Nuclear Physics - Controlling Medium Scale Nuclear Physics Experiments
| Nuclear physics experiments are centered around hardware - from power supplies to data acquisition hardware and everything in between. Not surprisingly, Erlang is a very natural fit for monitoring and controlling the hardware which is dedicated to those experiments. I will discuss two instances in which Erlang has been used to this purpose - one large scale and one medium scale experiment - and share stories and lessons learned on the battlefield of Physics. Talk objectives: I hope to demonstrate that Erlang is a very successful language in its "natural environment", which is controlling hardware. In this case, it is applied in an unusual field - Nuclear Physics - but is nonetheless an excellent fit. Target audience: Anybody who is interested in monitoring and controlling remote and often exotic hardware, or sensing with Erlang in general. Physicists (current or recovering) may also be interested in this particular application of the language. |
Genomu: A Concurrency-Oriented Database
| Genomu is a database implemented in Elixir (http://elixir-lang.org) and built on top of Riak Core stack that combines the idea of event sourcing, simple redis-like API and a conflict resolution technique based on log merging to enable highly fault tolerant operations. Talk objectives: To show Riak Core and Elixir can be used to build a database engine and explore what kind of opportunities and tradeoffs Genomu can offer. Target audience: Early adopters. Link to slides: https://dl.dropbox.com/u/923732/genomu-efsf13/index.html#/ |
| Active-once is a fantastic way of coping with 99% of flow- control problems in Erlang, but what can you do if you have to cope with unbounded input? The Heroku log router Logplex is responsible for collecting and directing the entire log output of the Heroku platform and all applications running on it. This talk will explain some of the tricks Logplex plays in order to stay ahead of the input volume. Talk objectives: Explain some (moderately crazy) Logplex tricks: raw tcp ports, bounded buffer processes and tcp connection management. Target audience: Erlang programmers building high-throughput tcp systems. |
| The BEAM is the standard Erlang implementation in use today. It was specially designed just to run Erlang. We will make a quick tour of the BEAM visiting the major sights to see what it looks like internally and how it functions. There will be no need of a towel on this short tour. Target audience: Developers and system architects who want to get a better understanding of how their applications will run. |
How OpenX Built a Scalable Global Digital Revenue Platform
| As a global leader in digital and mobile advertising revenue products and services, OpenX serves ads for many thousands of companies. At peak times, the OpenX Market (a pioneering Real-Time Bidding exchange) alone receives over 1 million bids per second. An additional challenge is that ads must be served in just a few hundred milliseconds. In 2012 OpenX served nearly 4 trillion ads, a near 300% growth from the 1 trillion the Company served in 2011. In order to scale and support this rapid growth OpenX has leveraged Erlang in many parts of its serving stack. In this talk, the architecture will be explored and the open source tools used to build, deploy, monitor and troubleshoot these systems will be discussed. Talk objectives: This talk will discuss the challenges of bringing Erlang into an organization, and how these challenges were met using various open source tools. The goal is for it to not just be the story of how OpenX came to be running Erlang on thousands of machines, but the tools that were used to get there. Target audience: Architects, Systems Engineers, Software Developers, DevOps As a global leader in digital and mobile advertising revenue products and services, OpenX serves ads for many thousands of companies. At peak times, the OpenX Market (a pioneering Real-Time Bidding exchange) alone receives over 1 million bids per second. An additional challenge is that ads must be served in just a few hundred milliseconds. In 2012 OpenX served nearly 4 trillion ads, a near 300% growth from the 1 trillion the Company served in 2011. In order to scale and support this rapid growth OpenX has leveraged Erlang in many parts of its serving stack. In this talk, the architecture will be explored and the open source tools used to build, deploy, monitor and troubleshoot these systems will be discussed. Talk objectives: This talk will discuss the challenges of bringing Erlang into an organization, and how these challenges were met using various open source tools. The goal is for it to not just be the story of how OpenX came to be running Erlang on thousands of machines, but the tools that were used to get there. Target audience: Architects, Systems Engineers, Software Developers, DevOps |
Introducing LeoFS - the Lion of Storage Systems
| To quote Ludwig Mies van der Rohe, "Less is more" and "God is in the details". This is the philosophy behind LeoFS, a DFS(Distributed File System) open source storage system developed and used by Rakuten, Japan's answer to Amazon and its #1 Shopping Site. LeoFS is a Web shaped object storage system that stores and serves many Rakuten’s image files. It is built to operate in highly distributed environments, such as the cloud, and consists of loosely connected nodes with no SPOF(single point of failure). Together, these nodes can be viewed as "ONE-Huge Storage". Yousuke Hara will be introducing LeoFS, and will describe how to use it to build a powerful web-shaped object storage system. He will also describe some of the business benefits and advantages. Oh, and he might mention it is written in Erlang, even if this is not the focus of the talk. Talk objectives: To give deep understanding of LeoFS. Target audience: * Web developers who want to use high-performance and high-reliability storage system, * Architects and project managers who want to discover highly reliable S3-compatible object storage system. |
Introduction to Elixir: a New Language on the VM
| Elixir is a programming language for the Erlang VM. Elixir provides a first class macro mechanism, supports polymorphism via protocols (similar to Clojure's) and many other features while keeping the functional aspects of Erlang used to build distributed, fault-tolerant applications. In this talk, José Valim will introduce Elixir and showcase some of its features, going into details about how they can be a useful part of developer's daily tools. Talk objectives: The talk aims to introduce Elixir and show case its main features. The attendees will leave the talk with a better understanding of the language goals and ready to start a project with Elixir. Target audience: This talk appeals and targets existing Erlang developers (with any level of experience). |
KVDB - A Database System for Configuration Data and Connected Devices
| Deciding to write your own database management system while implementing a commercial online service must surely be a grand example of hubris! Yet, this is what we did at Feuerlabs. We wanted a database framework that could work equally well on an embedded device as on the server side, with easy interoperability; we wanted to handle tree-structured data and a huge amount of persistent queues; and we needed transaction support. We are now releasing the result as part of our Open Source Exosense device programming framework. In this talk, we will give an overview of what KVDB can do, and where we are planning to take it. We will also describe some implementation details, some of which we think are particularly novel and Erlang-esque. Talk objectives: Introducing the Open Source DBMS KVDB, and giving an overview of its uses. Target audience: This talk is mainly for Erlang developers, who are either browsing for database systems, or interested in their design aspects. |
Kazoo: An Erlang-Based Elegant, Distributed Cloud Architecture for Communications
| This is a frank discussion about the benefits and challenges of scaling Erlang for massive communications applications in distributed environments. The talk will focus on how 2600hz uses Erlang, some of the Best practices we've learned while building our stack and how Erlang has really helped us get to market faster and easier. Kazoo leverages Erlang as a core component of the infrastructure and so we have developed some unique insight into how Erlang can be run at scale. In particular, we focus on maintaining complex systems in multiple geographic areas using some of the unique properties that Erlang allows. Talk objectives: This talk is intended to display and address Erlang at Scale. Erlang has been used for telecom systems since time immemorial, but the sheer distribution and open structure of Kazoo is different than what has come before. This talk aims to help folks understand why we chose Erlang, why we love it and how it gives us a competitive advantage. Target audience: This talk would appeal to folks in the Telecom industry, folks who build large systems and people that are curious about what the distributed world looks like at scale. |
LDB: The BigData In-Memory Database built with Erlang, C and LISP
| Would you believe it that you can feed data coming from various sources (even thousand different resources - like mobile devices) into a system, describe what information you want to extract in a few lines of code and then have all this information in your finger tips? In real time. While the system keeps running? LDB powers BugSense and analyses data coming from more than 200M mobile devices, in realtime. And even though it is In-Memory, data is actually stored in the hard-drive and replicated across other nodes using the awesome features of Erlang. With LDB we don’t run queries. We run algorithms due to a full blown custom LISP language written in C that has access to the same address space with the database. That means that you can search extremely fast for data and run complex algorithms insanely faster than any other solutions. |
Lager: Logging with Confidence
| Logging with Erlang can be fraught with peril. If it isn't a gen_server with a 10mb binary in its state crashing and OOMing the node as error_logger tries to print its state, it's your webserver crashing with 10,000 open connections with each one trying to log that it crashed. Or maybe you're writing an Erlang-powered product and don't want to scare your customers with scary, giant Erlang tuples in their log messages. Lager is Basho's answer to the problems it encountered shipping Riak with the default SASL logger. Lager's goals are to provide safe, reliable, customer friendly logging with familiar UNIX logfile semantics. Talk objectives: This talk will cover Lager's approach for solving the traditional problems with Erlang logging and cover some of the handy extra features it provides. Target audience: People who have been bitten by Erlang logging in the past, or anyone who is planning on deploying Erlang at scale and isn't aware of what can go wrong with logging. |
Lilith - Solving the Distributed Video Operations Challenge @ ESPN
| This talk will be about how John came to use Erlang for a system that requires a high-performance control plane and a system for handling long-running requests for video streaming. Talk objectives: Education. Specifically about the use of Websockets, riak_core and video processing in Erlang, and the comparative advantages/disadvantages of Erlang vs. node.js. Target audience: People wishing to understand Erlang's role in distributed systems design. Programmers wanting to know something about Erlang's FFI, riak_core and websockets. |
| A simple mechanism for tracking user location and finding others nearby is presented and contributed as open source. Geohashes expressed as integers padded with MSB=1 are used to store user location information with timestamps in Redis. An OTP application built on Mochiweb exposes a RESTful interface for setting user location information and finding neighboring users. If time permits, an integration with Mongoose may also be presented. Talk objectives: Location Matters. Users of websites should be able to find other users of that website who happen to be nearby. Target audience: Anyone running a website that intends to connect people to each other should have a location mechanism enabling people to find others nearby. (Life happens offline, too.) |
Locker: Consistent Distributed Locking
| Game servers built at Wooga uses Erlang processes to model users, worlds and connections. A locking mechanism was needed to ensure processes don't step on each other toes. This forms the backbone of distributed real-time multiplayer game servers at Wooga. The first attempt used Redis as a central serialization point as it offers a compare-and-swap operation. After hard work in the following generations of game servers, this single point of failure was the last one still standing. After long discussions and reading many papers, locker got started as a Friday hack. How hard could it be to create a consistent distributed locking system from scratch? Talk objectives: Share the journey from Friday hack to real production software. Target audience: Engineers interested in available systems, distributed systems, Riak, Zookeeper. |
| Monads (or "triples", or "standard constructions") were invented by Category Theorists around 1960, discovered as a tool for modularizing denotational semantics in 1991, and adopted by the Haskell community immediately thereafter. Now, after more than half a century, the programming community at large is finally ready to embrace monads. Scala, JavaScript, C#, Java, Closure, Erlang, Python, Ruby, Perl, PHP, you name a language and chances are that someone will have written a monad library for it. Yet many developers still consider monads as an esoteric feature, believe they have to do something with "side effects", or simply use them as a source of "clever" puns. In this talk. we will explain how every developer could, and probably should have, invented monads herself. In fact, we will present not one, not two, but at least 5 reasons for everyone with at most an AP Computer Science to experience extreme face palm for not having come up with monads themselves. This is the last talk you ever need to hear about monads. Afterwards, you will be able to elucidate a complete stranger about monads in less than 60 seconds. Guaranteed, or you get your money back. |
MongooseIM - Messaging that Scales
| MongooseIM provides a unique instant messaging solution when massive volume is required. The growth of social media communication has resulted in the fact that the ceiling is quickly reached when successful projects take off. MongooseIM secures performance and stability. In this talk I present lessons learnt when scaling messaging servers using common PC, cloud environments or a dedicated data centre. I demonstrate what bottlenecks we encountered and how to work around them in order to scale to millions of users. I also explain why MongooseIM and Erlang are a good baseline for building a custom messaging solution. Talk objectives: to educate about MongooseIM, which is built around proven technology Jabber/XMPP, as well as discuss how to build a messaging solution, which will scale to big numbers. Target audience: People who know Jabber/XMPP or are interested in messaging servers. |
Multimedia Galore: Sending JPGs and 3GPs and AMRs!
| A very popular feature of the WhatsApp Messenger smartphone application is the ability to send images, videos, and audio notes to the smartphones of friends, family members, and groups. As one of our 2012 server projects, we replaced our original multimedia system with an Erlang-based system. In this talk we will describe the architecture of the new system, some of the obstacles we encountered, and the scalability, functionality, and operability improvements that were achieved. Talk objectives: To describe the challenges and opportunities in building a scalable multimedia store-and-forward system using Erlang. Target audience: People interested in multimedia file serving using Erlang in a global-scale communications system. |
| This talk will cover the use of Erlang in the development of Neural Networks, and other types of evolving learning Universal Learning Network based intelligent agents. The talk will elaborate on why Erlang is a perfect fit for the development of distributed neural networks, the importance of Erlang's features to the field and future of Computational Intelligence, its role as the quintessential neural network programming language, and provide a case study of its use in the construction of distributed topology and parameter evolving universal learning network called DXNN. Talk objectives: To discuss the utility, applicability, and effectiveness of Erlang within the field of Computational Intelligence development, and to provide a case study of using Erlang to develop a scalable universal learning network. Target audience: Computational Intelligence researchers and enthusiasts, Erlang programmers and anyone that is generally interested in developing soft computing systems. |
Realtime Logging: Alerts, Hashtags and Live Log Feeds
| Popcorn is a cool new open source monitoring and logging platform that can provide the data needed to find and fix problems in a production environment. By incorporating live alerting, consolidated streaming, #topics and @identities into logging, Popcorn is able to take the power of server logs to a whole new level. Talk objectives: - Learn about how you can improve visibility and decrease your response time for production issues by using Popcorn, a new log management tool written in Erlang. - Learn how and why we built Popcorn and how it’s transformed our ability to both discover and diagnose issues. - Take away best practices that have provided us with new insights into how our servers are running. - Finally, learn how Popcorn works and how you can get involved in this open source project Target audience: SysOps/DevOps engineers and Erlang. developers Popcorn is a cool new open source monitoring and logging platform that can provide the data needed to find and fix problems in a production environment. By incorporating live alerting, consolidated streaming, #topics and @identities into logging, Popcorn is able to take the power of server logs to a whole new level. Talk objectives: - Learn about how you can improve visibility and decrease your response time for production issues by using Popcorn, a new log management tool written in Erlang. - Learn how and why we built Popcorn and how it’s transformed our ability to both discover and diagnose issues. - Take away best practices that have provided us with new insights into how our servers are running. - Finally, learn how Popcorn works and how you can get involved in this open source project Target audience: SysOps/DevOps engineers and Erlang. developers |
Refuge: Build a Decentralized Data Platform
| The Refuge project aims to build a decentralized and open-source data platform allowing you to share, manipulate and link any kind of data. This talk will highlight different features of the refuge platform built in Erlang, how we did them and why using Erlang for them: - coffer: a blob server able to store the data on different backends (FS, S3, Cloud files, ...) - blanket: a document-oriented database library able to use different backends (couch_btree obviously but also eleveldb, bitcask, hanoidb, ...) - beacon: the layer that handles the data nodes discovery (P2P) and different levels of trusts and authentication. - postie: the message processing system that promotes distributed and decentralized topologies without single point of failures (pretty similar to nsq but in Erlang) Talk objectives: Present all the aspects of OTP and Erlang used to build a data platform fully decentralized, fault tolerant, without single points of failures. This talk will also demo refuge working on embedded devices. Target audience: Anyone interested in building decentralized and distributed systems. |
S**t happens - Are you Prepared?
| Staying agile as businesses scale is tough and filled with conflicts – data models and processes become more rigid, while new use cases for data keep showing up. On top of this, your clients (and co-workers!) will find new ways to break things, ways that you could never have imagined in your wildest dreams. This is the domain of The Erlang Way - not just for your application and/or your systems, but for your entire organization. Join me as I take you through the application of the Erlang mindset to all aspects of your business (with pictures!) Talk objectives: Learn why loose coupling, agility, and fault-tolerance should be characteristics of every facet of your organization - not just your applications Target audience: People who work in groups of > 1 developer Staying agile as businesses scale is tough and filled with conflicts – data models and processes become more rigid, while new use cases for data keep showing up. On top of this, your clients (and co-workers!) will find new ways to break things, ways that you could never have imagined in your wildest dreams. This is the domain of The Erlang Way - not just for your application and/or your systems, but for your entire organization. Join me as I take you through the application of the Erlang mindset to all aspects of your business (with pictures!) Talk objectives: Learn why loose coupling, agility, and fault-tolerance should be characteristics of every facet of your organization - not just your applications Target audience: People who work in groups of > 1 developer |
Scaling Up from 1000 to 10 Nodes
| This is a story of how a large-scale data collection system that used to run on 1000 nodes was redesigned to run on only a handful of nodes. The highly concurrent soft real-time system presented in the talk is used for reliably collecting 10s of terabytes of data from 100s of thousands of sources every day. We will talk about various challenges of building and scaling the system in Erlang and speculate about advantages and limitations of Erlang stack for modern server-side development. Talk objectives: Share the experience of building and scaling a system in Erlang. Target audience: Anyone interested in using Erlang for building non-trivial systems. |
Services Platform: Experiments in OTP-Compliant Dataflow Programming
| Erlang Services Platform (Erlang/SP) is a library that is fully integrated and compliant with existing OTP libraries, but attempts to enable Dataflow Programming techniques. The library anticipates future chips with 100s - 10Ks of cores, allowing problems to be solved by organizing Cooperating Processes (co-ops) around Directed Acyclic Graphs (DAGs) as an augmentation to traditional gen_* system architectures Talk objectives: Compares and contrasts typical OTP programming techniques with Dataflow Programming approaches to demonstrate an alternative strategy for designing system architectures with Erlang. Specific code examples illustrate architectural tradeoffs. Target audience: Any developers of complex on-demand services implemented in Erlang/OTP, especially those hosted on multicore CPUs. Talk objectives: Compares and contrasts typical OTP programming techniques with Dataflow Programming approaches to demonstrate an alternative strategy for designing system architectures with Erlang. Specific code examples illustrate architectural tradeoffs. Target audience: Any developers of complex on-demand services implemented in Erlang/OTP, especially those hosted on multicore CPUs. |
| Describing the architecture of the abstraction layer we use for storage. The main properties of the platform are: - Massively scalable - A transparent layer for sharding - Works with any storage backend (MySQL, RIAK...) - Offers caching - Can be queried used native erlang or HTTP (JSON or ProtocolBuffers) Talk objectives: Showing the properties of the Spilgames Storage platform. Sharing some details of the internal implementation and the thinking process that led us to implementing this solution. Getting feedback about some of the concepts and design techniques used in this project. Target audience: Software architects, DB engineers, Backend Developers. |
Tailflow - An Openflow Controller Framework
| Tail-f's Network Control System (NCS) supports programming of network services on both traditional networks and Openflow networks. Tailflow is the part of NCS that supports Openflow. Inspired by the 20 year old ideas around micro-protocols, Tailflow introduces a concept called the 'flowlet'. Each flowlet encapsulates some minimum amount of logic in order to make it composable and easy to understand. By stacking flowlets on top of each other, the full functionality of an Openflow device is constructed, may it be an ordinary learning switch or a more complex NAT/FW device or, in principle, any other kind of networking device. This talk will describe the architecture of Tailflow and show how we, by using OpenVSwitch, created a simulated Data Center with many server racks, each server running many VMs. A Layer-2 virtual network is implemented as well as a NAT/FW for public access, all controlled by a Tailflow application. Talk objectives: Describe an approach for constructing Openflow control software. Target audience: - Network engineers. - People interested in Openflow and/or protocol design. - People interested in Software Defined Networks (SDN). |
Taking Back Embedded: The Erlang Embedded Framework
| Erlang was originally designed to control telephony switches at Ericsson which, by definition, are embedded systems. Somewhere along the line the application area changed dramatically and now Erlang is being used to tackle challenges which involve gratuitous amount of parallelism and “The Cloud”. The Internet of Things is the physical extension of cloud which describes how everyday objects around us will become sources of data that will transform our daily lives. Analysts forecast the number of Internet connected devices to reach 50 billion within the next decade, which signifies that we need to think of new ways to architect these new generation of connected devices. This talk will demonstrate how, by creating a layered architecture for hardware modules and partitioning up complex systems in smaller units, testing becomes easier, runtime errors are contained and the architecture becomes maintainable. Using Erlang processes as compositional units to describe these systems is a new proposal which stands out and challenges conventional approaches. Talk objectives: This talk aims to provide an overview of the current state of Erlang in the embedded domain and talk about our plans to help speed up the adoption rate of Erlang in embedded projects. Target audience: Hardware and software engineers interested in discovering new tools and methodologies for tackling the next generation of connected embedded devices. |
The How and Why of Fitting Things Together
| Software is difficult because the parts don't fit together. Why is this? Can we do anything about this? And what's this got to do with Erlang? Come to my talk and you'll find out! |
| Le Cowboy nouveau est arrivé ! Cowboy 0.8 brought many changes in the way you use Cowboy to build powerful web applications. This talk will detail what's new, explain why we made these decisions and what to expect in the future. Talk objectives: If you don't know Cowboy: understand what it is and what it isn't, and figure out how to use it to harness the power of the soft real time web. If you already know Cowboy: understand what changed in Cowboy, what remains to be done and how you can help. Target audience: * Web developers who want to use Erlang to build high performance web applications * Erlang developers who need the best web technology at their disposal * Architects and project managers who want to discover how easy it can be to do soft real time web applications |
The Polyglot Rabbit and the Big Fluffy Cloud
| RabbitMQ (http://www.rabbitmq.com) is a popular 100% Erlang-based Open Source message queuing system that implements the Advanced Message Queuing Protocol (AMQP). It has been estimated that there are presently some 30,000 production deployments of RabbitMQ across the globe, and this number is growing rapidly. Most of these deployments are business-critical, underpinning everything from internet-based pizza ordering systems through to providing the central nervous system for OpenStack-based cloud deployments. RabbitMQ currently supports versions 0.8.0 and 0.9.1 of AMQP and will soon also provide support for 1.0. However, a somewhat overlooked capability of RabbitMQ is its ability to also readily provide support via a flexible plugin architecture for a variety other popular Open Source message queuing protocols, including STOMP, MQTT, ZeroMQ, and RESTful messaging via the RabbitHub plugin. Most good message queuing protocols share many features in common; however some are better suited to a particular set of use cases than others. This ability of RabbitMQ to be able to seamlessly receive and propagate messages simultaneously via multiple protocols is an extremely powerful facility, and one that affords great flexibility. For example, it means that it is possible to use the most appropriate protocol for a particular function or to simultaneously use different protocols to disseminate the same data to different types of users via the most appropriate protocol without having to develop and maintain any separate gateway components. In this talk the presenter will discuss the multi-protocol features of RabbitMQ and how the capabilities of Erlang have been utilised to implement the powerful RabbitMQ plugin architecture. An overview of the plugin architecture will be presented along with examples of its use to implement a robust and highly scalable multi-protocol Open Source cloud-based messaging hub. Talk objectives: To describe the multi-protocol capabilities of RabbitMQ, how Erlang has been used to efficiently implement these capabilities, and how these capabilities can be leveraged to implement a powerful multi-protocol Open Source cloud-based messaging solution. Target audience: General audience (there will be some content of a technical nature, but this will not be integral to the talk). |
The Pooler Story: How I Learned OTP by Writing a Connection Pool
| Pooler is a generic process pooling OTP application. It was originally designed as a connection pool for Riak and was later repurposed to pool Postgresql connections in Opscode's Hosted Chef web service. This is the story of the design evolution of pooler and how it helped me understand OTP. The primitives for process linking in Erlang make building an exclusive access pool easy. The task of composing those primitives in a way that follows the OTP design principles is less straight forward for the beginner. In this talk I'll describe the journey from simple gen_server to complete OTP application and highlight some lessons learned when running the application under load and integrating it into OTP releases. Talk objectives: - Introduce pooler, a generic process pooling application - Highlight how the primitives available in OTP make writing a connection pool easy. - Admit that even in Erlang, writing a connection pool is still hard. - Demonstrate standard OTP features such as gen_servers, supervisors, and monitoring in the context of an application used in production. Target audience: Erlang developers interested in learning or teaching OTP principles. Those looking for a connection pool. |
WebSockets, RabbitMQ & Erlang @ the Huffington Post
| In this talk, you will learn about the architecture and setup that the Huffington Post built to power their realtime websocket based commenting framework. Leveraging RabbitMQ, SockJS, Cowboy and Erlang, they were able to build and design a highly scalable, customizable solution to power realtime comments. You will learn about some of the pitfalls they hit, as well as some of the improvements they would like to make to take their platform to the next level. Talk objectives: - Understanding why we chose to go the Erlang and SockJS route for our commenting platform - How we integrate RabbitMQ routing to power our subscription architecture - How we manage subscriptions for realtime ""channels"" - Extending the framework into an open source solution - Enabling other mechanisms for publishing comments outside of RabbitMQ Target audience: For people interested in leveraging Erlang to power a realtime capability on their web applications, this talk will provide oversight into what we learned in the last year, how the pieces of the architecture fit together and how you might customize a websocket solution to suit your needs. |
| The Erlang/OTP team has been investigating and prototyping a new built in datatype "maps" for a while now. This talk will give you insight in the journey towards the current status when it comes to "better records" with atoms as keys and with more general "maps" with any terms as keys. Talk objectives: Broaden the knowledge about what the Erlang "maps" will be like. Target audience: SW developers with interest in programming language design and virtual machine design. |
| One of the most common questions on Stack Exchange is "Why does Erlang syntax resemble Prolog with asphyxia?" [1] In this talk, Garrett will obliterate the notion that Erlang syntax is weird or that you can't use it to write beautiful code. The plain truth is that it's easier to write beautiful code in Erlang than it is in most other languages. Garrett will describe methods for writing luscious, gorgeous, easy to read and maintain Erlang programs. He will cover topics such as: - API design - Function and variable names - Proper use of case and if expressions - Managing complex data structures - Code refactoring - Common functional patterns Garrett will draw from the coding styles of the Erlang masters as well as his own experience an Erlang programmer and instructor. He will argue will a zeal bordering on religious fervor, chiding those with different views. [2] [1] In fact, not a common question on Stack Exchange [2] No one will be chided -- the presentation will be quite congenial Talk objectives: To help Erlang developers write better code. Target audience: Erlang developers and anyone interested in functional design and coding style. Link to slides: http://www.gar1t.com/presentations/factory-sf-2013/#slide1 |