Empirical Methods in Software Engineering (01OPJIU)

PhD Course - 2013

News

• A template for the final report is available (also with LaTeX source files)
• Location of lectures: Aula C In the basement, follow directions for Aula 14
• Previous years' editions: 2012 and 2011

Schedule

1. Tue 19/11: 9.00 - 13.00 - Introduction, Overview of Empirical Methods in Software Engineering
2. Tue 26/11: 9.00 - 13.00 - The experimental process
3. Wed 27/11: 9.00 - 13.00 - Systematic Literature Reviews
4. Tue 3/12: 9.00 - 13.00 - Software Measurement
   Wed 4/12: 14.00 - 18.00 (Sala Riunioni 4) - Software Measurement
5. Tue 10/12: 9.00 - 13.00 - Data Analysis
6. Wed 11/12: 14.00 - 18.00 - Data Analysis
7. Tue 17/12: 9.00 - 13.00 - Surveys

Material

• Introduction: slides
• Overview: slides
  1. Screencast
• Experimental process: slides
  1. Screencast (part 1)
  2. Screencast (part 2)
  3. Screencast (part 2)
  • Sample worksheet for the Binomial test
• Systematic Literature Reviews: slides
  1. Screencast (part 1)
  2. Screencast (part 2)
• Software measurement: slides
  1. Screencast (part 1)
  2. Screencast (part 2)
• Data analysis: slides
  1. Screencast (part 1)
  2. Screencast (part 2)
  3. Screencast (part 3)
  • Paper:
    Ricca F., Di Penta M., Torchiano M., Tonella P., Ceccato M., Visaggio C. A. "Are Fit Tables Really Talking? A Series of Experiments to Understand whether Fit Tables are Useful during Evolution Tasks" IEEE International Conference on Software Engineering, Leipzig May 10-18, pp. 261-271, 2008
  • Raw data
  • Sample session.R
• Surveys: slides
  1. Screencast (part 1)
  2. Screencast (part 2)

Goals

• desing, plan, and conduct experimentation (and empirical validation in general) mainly in the context of software engineering, but also in other disciplines of Computer Science;
• perform data collection from all the different sources available during all the range of the software development lifecycle: text, software repositories, human subject observation (developers);
• build statistical models based on the collected data from software products, software processes and experiments

Basic references

• Koen, Billy Vaughn. (2003). Discussion of The Method: Conducting the Engineer's Approach to Problem Solving. New York: Oxford University Press
• Barbara A. Kitchenham, Tore Dyba, and Magne Jorgensen. 2004. Evidence-Based Software Engineering. In Proceedings of the 26th International Conference on Software Engineering (ICSE '04). IEEE Computer Society, Washington, DC, USA, 273-281.
• Basili, V. 1995. The Experience Factory and Its Relationship to Other Quality Approaches. In Advances in Computers, vol 41. Academic Press, 65-82
• Potts, C. 1993. Software-engineering research revisited. In IEEE Software, vol. 10(5), September, IEEE Computer Society, 19-28.
• Tonella P., Torchiano, M., Du Bois, B., Systa, T. 2007. Empirical studies in reverse engineering: state of the art and future trends. In Empirical Software Engineering, Vol. 12(5), Springer, 551-571.
• Glaser, B, Strauss, A., The Discovery of Grounded Theory: Strategies for Qualitative Research, Aldine de Gruyter, NY, 1967
• Wohlin, C., Runeson, P., Hst, M., Ohlsson, M. C., Regnell, B., & Wessln, A. (2012). Experimentation in software engineering. Springer Publishing Company, Incorporated.
• Basili, Victor; Gianluigi Caldiera, H. Dieter Rombach (1994). "The Goal Question Metric Approach"
• Van Solingen, Rini; Egon Berghout (1999). The Goal/Question/Metric Method. McGraw-Hill Education.
• Kitchenham, B. (2004). Procedures for Performing Systematic Reviews. Joint Technical Report TR/SE-0401.
• Kitchenham, B. and Charters, S. (2007). Guidelines for performing Systematic Literature Reviews in Software Engineering. Version 2.3 EBSE-2007-01.
• Brereton et al., (2007). Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software, 80, 571-583.
• Kitchenham, B., Brereton, P., Budgen,D., Turner,M., Bailey, J., Linkman, S. (2009). Systematic literature reviews in software engineering - A systematic literature review. Information and Software Technology 51, 7-15.
• Kitchenham, B. e tal.(2010). Literature reviews in software engineering - a tertiary study, Information and Software Technology 52 (8) 792-805.
• Fabio Q.B. da Silva, André L.M. Santos, Sérgio Soares, A. César C. Franca, Cleviton V.F. (2011). Six Years of Systematic Literature Reviews in Software Engineering: An Updated Tertiary Study Information and Software Technology 53(9) 899-913.

Exam