Embedded Linux Conference & OpenIoT Summit North America 2018

Embedded Linux drives an every-increasing number of appliances in many domains and applications, some even real-time and/or safety critical. Traditional quality assurance of such systems is based on testing and formal verification, but the huge amount of code and the rapid dynamics of the Linux ecosystem, as well as fundamental limitations of formal methods make these approaches unsatisfactory.

Statistical quality assurance for reliability, error rates, maximal latencies etc. is needed. We will discuss current best practises, how to design and run automated statistical tests that capture relevant information, and how to properly evaluate the resulting data. Practical real-world examples and recipes are played through using the open source R language. Most importantly, we identify common mistakes in (over-)interpreting statistical results and predictions that may eventually harm people.