Towards certifying the testing process of open-source software: New challenges or old methodologies?

To improve both the quality and the trustworthiness perception of Open Source Software (OSS) products, we introduce the new idea of certifying the testing process of an OSS system. While the global certification of an OSS product is an emerging research field, the idea of certifying only its testing process has never been studied, conversely to the case of Closed Source Software (CSS) products. The certification of the testing process has a twofold goal: simplify the process of testing OSS products by guiding developers in identifying the proper testing strategies and the limitations of their existing testing plans; simplify the selection of equivalent OSS and CSS products by evaluating the certificates released by the companies. Specifically, in this paper we discuss 1) a set of issues, inherent to OSS, that must be taken into account when testing the OSS product; 2) a preliminary methodology that suggests how to certificate the testing process of OSS products; 3) the BusyBox case study that shows how our idea can be applied to real-life OSS.