Empirical Methods in Software Engineering (01OPJIU)

PhD Course - 2014

Marco Torchiano
marco.torchiano AT polito.it
tel: 011 564-7088


  • Location of lectures: Classroom C - In the basement, follow directions for Classroom 14
  • Previous years' editions: 2013, 2012, and 2011


Schedule with tentative topics
  1. Tue 11 Nov, 14:00 - Introduction to the course
  2. Tue 18 Nov, 14:00Fri 14 Nov, 9:00 - Systematic Literature Reviews
  3. Fri 14 Nov, 9:00Tue 18 Nov, 14:00 - Experimental process
  4. Fri 21 Nov, 9:00 - Software Measurement
  5. Tue 2 Dec, 9:00 - Data visualization
  6. Tue 9 Dec, 9:00 - Data analysis
  7. Fri 12 Dec, 9:00 - Population Survey Data analysis
  8. Tue 16 Dec, 9:00 - Final presentation Population Surveys
All lectures will last approximately 3 hours and half.


The slides and screencasts of the lectures are availabale as a podcast.

Additional support materials:


Introduce methods and tools to:
  • desing, plan, and conduct experimentation (and empirical validation in general) mainly in the context of software engineering, but also in other disciplines of Computer Science;
  • perform data collection from all the different sources available during all the range of the software development lifecycle: text, software repositories, human subject observation (developers);
  • build statistical models based on the collected data from software products, software processes and experiments

Basic references

  • Koen, Billy Vaughn. (2003). Discussion of The Method: Conducting the Engineer's Approach to Problem Solving. New York: Oxford University Press
  • Barbara A. Kitchenham, Tore Dyba, and Magne Jorgensen. 2004. Evidence-Based Software Engineering. In Proceedings of the 26th International Conference on Software Engineering (ICSE '04). IEEE Computer Society, Washington, DC, USA, 273-281.
  • Basili, V. 1995. The Experience Factory and Its Relationship to Other Quality Approaches. In Advances in Computers, vol 41. Academic Press, 65-82
  • Potts, C. 1993. Software-engineering research revisited. In IEEE Software, vol. 10(5), September, IEEE Computer Society, 19-28.
  • Tonella P., Torchiano, M., Du Bois, B., Systa, T. 2007. Empirical studies in reverse engineering: state of the art and future trends. In Empirical Software Engineering, Vol. 12(5), Springer, 551-571.
  • Glaser, B, Strauss, A., The Discovery of Grounded Theory: Strategies for Qualitative Research, Aldine de Gruyter, NY, 1967
  • Wohlin, C., Runeson, P., Hst, M., Ohlsson, M. C., Regnell, B., & Wessln, A. (2012). Experimentation in software engineering. Springer Publishing Company, Incorporated.
  • Basili, Victor; Gianluigi Caldiera, H. Dieter Rombach (1994). "The Goal Question Metric Approach"
  • Van Solingen, Rini; Egon Berghout (1999). The Goal/Question/Metric Method. McGraw-Hill Education.
  • Kitchenham, B. (2004). Procedures for Performing Systematic Reviews. Joint Technical Report TR/SE-0401.
  • Kitchenham, B. and Charters, S. (2007). Guidelines for performing Systematic Literature Reviews in Software Engineering. Version 2.3 EBSE-2007-01.
  • Brereton et al., (2007). Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software, 80, 571-583.
  • Kitchenham, B., Brereton, P., Budgen,D., Turner,M., Bailey, J., Linkman, S. (2009). Systematic literature reviews in software engineering - A systematic literature review. Information and Software Technology 51, 7-15.
  • Kitchenham, B. e tal.(2010). Literature reviews in software engineering - a tertiary study, Information and Software Technology 52 (8) 792-805.
  • Fabio Q.B. da Silva, André L.M. Santos, Sérgio Soares, A. César C. Franca, Cleviton V.F. (2011). Six Years of Systematic Literature Reviews in Software Engineering: An Updated Tertiary Study Information and Software Technology 53(9) 899-913.


The exam typically consists in analyzing data either from a small experiment or observational study, or conducting a SLR.

A template for the final report is available (also with LaTeX source files)