Testing Asynchronous APIs With QuickCheck

It is a challenge to write test cases for software with asynchronous operations, i.e. operations that return immediately, but for which the result is only observable after a while. Writing tests with long sleep statements is error prone. Polling after each API call to observe the expected result of that call does not exhibit the realistic usage scenarios in which several API calls are spawning work in parallel. Writing sufficiently many meaningful manual tests is considered almost impossible, using QuickCheck for generating the tests seems the better choice. We describe how we used QuickCheck for an industrial cloud orchestration system used to manage VMs in the cloud. We generate random calls to the asynchronous API calls and add some poll operations at certain random moments in order to get the system in a state in which something interesting has happened. The approach is general in nature and can be applied to many different asynchronous systems. It is an efficient and effective way to test asynchronous web services. The test code contains less than 1% of the code of the project. Nevertheless, it helped to find 25 errors in a system that was already in production: about 19 lines of QuickCheck code per error found.

Talk objectives:

Get insight in using QuickCheck for testing complex software systems.

Target audience:

Erlang and Elixir developers.

Slides
Video

Training: QuickCheck

More details on this training course can be found here.

About Thomas

Thomas Arts, co-founder and CTO of Quviq, has worked at Ericsson's computer Science Lab in the nineties. Together with Joe Armstrong, Simon Marlow and Phil Wadler, he tried to build a type system for Erlang. That being of little practical use, he later worked on verification tools for Erlang. After a short return to academia as professor at Chalmers, he is now fully devoted to amazing testing tools.

GitHub: Quviq

Twitter: @quviq

Back to conference page