Step-By-Step Guide to Building an Application Analytics System

In this talk, we will present a simple and efficient system we built at WhatsApp for analyzing performance data collected from over 300 million mobile clients.

The system has several main functions:

- collect and store 100s of billions events per day
- run fast analytic queries over collected data using a SQL-like
language
- provide interface for visualizing data using time series, histograms
and other types of charts
- support flexible event schema with arbitrary number of dimensions

Talk objectives:

The goal of the talk is to highlight architectural choices and easily accessible tools that can help to solve this problem at any scale, regardless of whether it is a single-node or a multi- data center operation.

Target audience:

Anyone who is interested in practical and elegant OLAP solutions. If you routinely grep/awk/sed/perl your application logs or you can't do this any more because data volume makes it impractical this talk is for you.

Slides
Video

About Anton

Anton Lavrik (@alavrik) is a software engineer at WhatsApp where he builds custom analytics systems.

He is also the author of the Piqi project which goal is to redefine the way people think about structured data.

Back to conference page