Empirical Methods in Software Engineering (01OPJIU)

PhD Course - 2015

Marco Torchiano
marco.torchiano AT polito.it
tel: 011 564-7088



Schedule with tentative topics
  1. Tue 10 Nov, 9:00 - Introduction to the course + Intro Exp. Process
  2. Fri 13 Nov, 9:00 - Experimental process
  3. Wed 18 Nov, 9:00 - Software Measurement
  4. Fri 20 Nov, 9:00 - Data analysis
  5. Tue 24 Nov, 9:00 - Data visualization
  6. Fri 27 Nov, 9:00 - Systematic Literature Reviews
  7. Fri 11 Dec, 9:00 - Population surveys
All lectures will last approximately 3 hours and half.


The slides and screencasts of the lectures are availabale as a podcast.

Additional support materials:


Introduce methods and tools to:
  • desing, plan, and conduct experimentation (and empirical validation in general) mainly in the context of software engineering, but also in other disciplines of Computer Science;
  • perform data collection from all the different sources available during all the range of the software development lifecycle: text, software repositories, human subject observation (developers);
  • build statistical models based on the collected data from software products, software processes and experiments

Basic references

  • Koen, Billy Vaughn. (2003). Discussion of The Method: Conducting the Engineer's Approach to Problem Solving. New York: Oxford University Press
  • Barbara A. Kitchenham, Tore Dyba, and Magne Jorgensen. 2004. Evidence-Based Software Engineering. In Proceedings of the 26th International Conference on Software Engineering (ICSE '04). IEEE Computer Society, Washington, DC, USA, 273-281.
  • Basili, V. 1995. The Experience Factory and Its Relationship to Other Quality Approaches. In Advances in Computers, vol 41. Academic Press, 65-82
  • Potts, C. 1993. Software-engineering research revisited. In IEEE Software, vol. 10(5), September, IEEE Computer Society, 19-28.
  • Tonella P., Torchiano, M., Du Bois, B., Systa, T. 2007. Empirical studies in reverse engineering: state of the art and future trends. In Empirical Software Engineering, Vol. 12(5), Springer, 551-571.
  • Glaser, B, Strauss, A., The Discovery of Grounded Theory: Strategies for Qualitative Research, Aldine de Gruyter, NY, 1967
  • Wohlin, C., Runeson, P., Hst, M., Ohlsson, M. C., Regnell, B., & Wessln, A. (2012). Experimentation in software engineering. Springer Publishing Company, Incorporated.
  • Basili, Victor; Gianluigi Caldiera, H. Dieter Rombach (1994). "The Goal Question Metric Approach"
  • Van Solingen, Rini; Egon Berghout (1999). The Goal/Question/Metric Method. McGraw-Hill Education.
  • Kitchenham, B. (2004). Procedures for Performing Systematic Reviews. Joint Technical Report TR/SE-0401.
  • Kitchenham, B. and Charters, S. (2007). Guidelines for performing Systematic Literature Reviews in Software Engineering. Version 2.3 EBSE-2007-01.
  • Brereton et al., (2007). Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software, 80, 571-583.
  • Kitchenham, B., Brereton, P., Budgen,D., Turner,M., Bailey, J., Linkman, S. (2009). Systematic literature reviews in software engineering - A systematic literature review. Information and Software Technology 51, 7-15.
  • Kitchenham, B. e tal.(2010). Literature reviews in software engineering - a tertiary study, Information and Software Technology 52 (8) 792-805.
  • Fabio Q.B. da Silva, André L.M. Santos, Sérgio Soares, A. César C. Franca, Cleviton V.F. (2011). Six Years of Systematic Literature Reviews in Software Engineering: An Updated Tertiary Study Information and Software Technology 53(9) 899-913.


The exam typically consists in analyzing data either from a small experiment or observational study, or conducting a SLR.

A template for the final report is available (also with LaTeX source files)