Empirical Methods in Software Engineering (01QTFIU)

PhD Course - 2019/20

News

• A template for the final report is available (also with LaTeX source files)
• A CSV file containing the results of a sample survey is available
• The lecture of Fri 20 Dec has been canceled, the new schedule is available
• Location of lectures: Classroom C - In the basement, follow directions for Classroom 14
• Previous years' editions: 2017, 2015 2014, 2013, 2012, and 2011

Schedule

1. Fri 13 Dec, 9:30 - Introduction to the course

Fri 20 Dec, 9:30 - Experimental process


2. Fri 10 Jan, 9:30 - Experimental process
3. Tue 14 Jan, 9:30 - Systematic Literature Reviews
4. Fri 17 Jan, 9:30 - Systematic Literature Reviews (Lab)
5. Tue 21 Jan, 9:30 - Data Visualization
6. Fri 24 Jan, 14:00 - Data visualization (lab)
7. Tue 28 Jan, 9:00 - Data analysis
8. Fri 31 Jan, 14:30 - Population Survey

Materials

Additional support materials:

• The data analysis lecture will make us of R and R Studio
  1. Download R here
  2. Download R Studio Desktop here
  Note: first you need to install R then RStudio.

  Data useed in the lecture are available as a zip file

• A sample analysis session that uses the data (CSV file) from a simple questionnaire
• Experimental process running example: Assessing the Effect of Screen Mockups on the Comprehension of Functional Requirements
• Sample worksheet for the Binomial test

Goals

• desing, plan, and conduct experimentation (and empirical validation in general) mainly in the context of software engineering, but also in other disciplines of Computer Science;
• perform data collection from all the different sources available during all the range of the software development lifecycle: text, software repositories, human subject observation (developers);
• build statistical models based on the collected data from software products, software processes and experiments

Basic references

• Koen, Billy Vaughn. (2003). Discussion of The Method: Conducting the Engineer's Approach to Problem Solving. New York: Oxford University Press
• Barbara A. Kitchenham, Tore Dyba, and Magne Jorgensen. 2004. Evidence-Based Software Engineering. In Proceedings of the 26th International Conference on Software Engineering (ICSE '04). IEEE Computer Society, Washington, DC, USA, 273-281.
• Basili, V. 1995. The Experience Factory and Its Relationship to Other Quality Approaches. In Advances in Computers, vol 41. Academic Press, 65-82
• Potts, C. 1993. Software-engineering research revisited. In IEEE Software, vol. 10(5), September, IEEE Computer Society, 19-28.
• Tonella P., Torchiano, M., Du Bois, B., Systa, T. 2007. Empirical studies in reverse engineering: state of the art and future trends. In Empirical Software Engineering, Vol. 12(5), Springer, 551-571.
• Glaser, B, Strauss, A., The Discovery of Grounded Theory: Strategies for Qualitative Research, Aldine de Gruyter, NY, 1967
• Wohlin, C., Runeson, P., Hst, M., Ohlsson, M. C., Regnell, B., & Wessln, A. (2012). Experimentation in software engineering. Springer Publishing Company, Incorporated.
• Basili, Victor; Gianluigi Caldiera, H. Dieter Rombach (1994). "The Goal Question Metric Approach"
• Van Solingen, Rini; Egon Berghout (1999). The Goal/Question/Metric Method. McGraw-Hill Education.
• Kitchenham, B. (2004). Procedures for Performing Systematic Reviews. Joint Technical Report TR/SE-0401.
• Kitchenham, B. and Charters, S. (2007). Guidelines for performing Systematic Literature Reviews in Software Engineering. Version 2.3 EBSE-2007-01.
• Brereton et al., (2007). Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software, 80, 571-583.
• Kitchenham, B., Brereton, P., Budgen,D., Turner,M., Bailey, J., Linkman, S. (2009). Systematic literature reviews in software engineering - A systematic literature review. Information and Software Technology 51, 7-15.
• Kitchenham, B. e tal.(2010). Literature reviews in software engineering - a tertiary study, Information and Software Technology 52 (8) 792-805.
• Fabio Q.B. da Silva, André L.M. Santos, Sérgio Soares, A. César C. Franca, Cleviton V.F. (2011). Six Years of Systematic Literature Reviews in Software Engineering: An Updated Tertiary Study Information and Software Technology 53(9) 899-913.

Exam

The exam typically consists in analyzing data either from a small experiment or observational study, or conducting a SLR.

The topic of the exam has to be agreed with the teacher.