Build Scientific Computing Infrastructure with Rebar3 and Docker

The Cytoscape Project (www.cytoscape.org) provides a desktop network analysis tool that has helped pioneer the field of bioinformatics. With the addition of an app store, we have also enabled researches to share their algorithms and processes in the form of app plugins for the desktop. The next evolution of Cytoscape goes one step further in collaboration by creating one large distributed network of services and and clients over our new Cyber-Infrastructure layer. We have recreated our desktop from scratch as a web application that consists of components connected to services through agents which other projects can embed into their own sites. Researchers can also write services that run in Docker containers that connect automatically to the agent network. Erlang holds a critical spot in this infrastructure, we produced ELSA, our ErLang Submit Agent in the language. Elsa forms a global distributed network by allowing nodes to register with other nodes just as a service can register with a node. This creates a relay network where clients have requests relayed to a node that provides the service they are looking for. Agents can run anywhere, and because they use mnesia as their data store, they can go down for maintenance and still preserve agent registration and complete request data. Elsa strives to remain invisible to our users, allowing them to focus their time on the more important matter of their research. Erlang allows our researchers who write services to remain oblivious to the large network of relay nodes that handle connecting the clients that need the algorithms provided by these services by insulating the two with a layer of fault tolerance and scalability. We wish to discuss the various features of Erlang that helped us achieve our goals, our plans for the future and how to become a host for our services, as well as provide examples of real research that has been furthered by our cyber infrastructure.

Talk objectives:

Every living organism on the planet has an interest in curing disease. With a few teams across the globe working on this project, we currently enable researches to save precious time in their research methods, and provide insights into tough medical problems needed to develop cures. This talk focuses on exciting developers about where Erlang has taken us, and how they can participate in the race against Cancer, Malaria, Alzheimer, and the myriad of illnesses that surely affect us all. We welcome contributors and altruists who want to run our services or Erlang agent on their own hardware, or who wish to contribute to our open source projects on Github.

Target audience:

Anyone who wants to know more about how Erlang helps researchers study and cure disease with genomics and computational biology. Developers looking for patterns to marry Docker and Erlang. Users working on collaborative open source projects, looking for tools to build SOA supercomputing tasks over the internet, or are interested in how Erlang and mnesia scale when routing computationally heavy tasks across the globe.

Slides
Video

About Eric

I'm a recent graduate with a degree in Computer Science. I moved from a role in Cloud and Devops at General Electric to UCSD's Ideker lab, where I work with a team of computational biologists and software architects to create tools that enable research in the fields of bioinformatics and general medicine. I fell in love with Prolog in school, and discovered Erlang shortly thereafter. I brought my knowledge and passion for Erlang to the lab, and have made converts out of researchers who primarily focus on statistical languages. I am also an early adopter of Docker in production from my time at GE, and helped build distributed infrastructure in Erlang to run our containers across our compute resources.

GitHub: ericsage

Twitter: @sageProgramming

Back to conference page