Cuneiform, DSLs For Scientific Workflows

Complexity of scientific workflow logic and the complexity of the underlying architectures will quickly become a nightmare if those two concerns aren't properly separated.   Scientific workflow DSLs to the rescue!   One of the newest and hottest ones out there is Cuneiform, first ever functional scientific workflow DSL.   Currently Cuneiform is being rewritten from Java to Erlang, a much better suited language for the job.  

Cuneiform supports HTCondor as one of its possible backends. We'll also talk about HTCondor, oldie but very goodie job scheduling and orchestrating architecture created at UW-Madison and how we use it for LiDAR image processing at HERE. 

Talk objectives:

* Demonstrate the challenges of HTC architectures.

* Demonstrate the importance and the benefits of using scientific workflow DSLs. 

* Demonstrate the use of Cuneiform, a functional workflow DSL written in Erlang.

Target audience:

* Software architects dealing with HTC architectures. 

* Programmers interested in creating and designing their own DSLs.

Slides

About Irina

Irina is a Sr. Architect at HERE, having fun with the challenges of HTC. 

GitHub: iguberman

Twitter: @irina_guberman

Back to conference page