Welcome and New Introduction

Chair: Dieter Rombach

ISERN is open to academic and industrial groups world-wide that are active in empirical software engineering research and willing to adopt the empirical research framework. ISERN members are pairs of organization and contact person. If the contact person leaves the organization, the organization must reapply for membership. Interested organizations may apply by sending an electronic proposal to "ISERN at informatik.uni-kl.de" describing their past experience in empirical software engineering research as well as their expectations from a future ISERN membership. Candidates will be invited to observe the ISERN Meeting following their application.

The goal of the session is to facilitate the membership application process by giving an opportunity for candidates to present their research and for observers to introduce themselves. Membership is granted according to a 3-step procedure:
1) Attending as invited observer at an annual ISERN meeting.
2) Attending as invited candidate at the following ISERN meeting giving a presentation. Membership is granted if a two-thirds majority of current members approve the application in an email voting after the meeting.
3) Attending as a full ISERN member at following meetings.

Current members present contact/affiliation changes (give a 2 min presentation each)

Reading List

Chairs: Dietmar Pfahl, Dan Port

This session is a continuation of last year’s 'ISERN reading list' plenary workshop session. The ISERN reading list is supposed to establish an agreed set of core readings for ISERN members. Last year, Per Runeson and Sira Vega presented content and structure of the ISERN reading list, and they collected feedback from the plenary.
The goal of the session is to present and discuss a proposal on the future evolution of the ISERN reading list, the externalization of a core reading list on empirical software engineering methodology (as part of the Wikipedia page on Empirical Software Engineering), and a related governance policy. The session will be structured as follows:
- Presentation of ‘ISERN Reading List’ and plans for integration in the ISERN web-page as a wiki with read/write access for all ISERN members, ‘ESE core literature in Wikipedia’, ‘Governance Policy’ (20 min)
- Plenary discussion (20 min)
- Distribution and collection of feedback questionnaire (20 min)
Based on the discussion and analysis of the feedback questionnaire, consultation with the ISERN Steering Committee on implementation of the proposed plans, i.e., the ISERN reading list governance policy and wiki.
The participants will receive documents via the ISERN mailing list before the ISERN meeting and should have read these documents before the session starts.

Drawing the ISERN Community

Chairs: Mike Barker, Barbara Russo

Research Affinity Diagram, want ads and citation graph

Ongoing Collaborations

Chair: Stefan Biffl

Searching for Collaborations

Chair: Ayse Bener

Open Space

This is a space for informal meetings. A list of announced meeting is available here

Students vs practitioners

Chairs: Natalia Juristo, Markku Oivo

For years we have wonder how good students represent practitioners in SE experiments. Some colleagues tend to think that students for sure perform worse than practitioners. But students might have some strengths as experimental subjects (more motivation, easiness to learn, higher commitment, ...). To better understand the generalization to practitioners that we can expect from experiments with students we need to learn more about this issue.

Session goals:

• How can we proceed for learning more about type of experimental subjects in SE experiments and the kind of generalization we can expect for each type?
• Identify ISERN members’ experience about the topic
• Identify interested contributors for researching on understanding more in depth the differences between professionals and students

Development of the session: We aim to have an open discussion. We will start off with some motivation for the session. Then 4 panelists (Claes Wohlin, J$uuml;rgen M$uuml;nch, Andreas Jedlitschka and Burak Turhan) will present their view in a short 5 minutes talk. The panelists will first comment on where their experience come from and then present their position on the topic. Which allows about 30 minutes for discussion with the audience.

Background and Recommended reading list:
M. Höst, B. Regnell, and C. Wohlin, ―Using Students as Subjects — A Comparative Study of Students and Professionals in Lead-Time Impact Assessment,‖ Empir. Softw. Eng., vol. 5, no. 3, pp. 201–214, 2000.
P. Berander, ―Using students as subjects in requirements prioritization,‖ Proceedings. 2004 Int. Symp. Empir. Softw. Eng. 2004. ISESE ’04., pp. 167–176, 2004.
M. Svahnberg, A. Aurum, and C. Wohlin, ―Using students as subjects - an empirical evaluation,‖ in Proceedings of the Second ACMIEEE international symposium on Empirical software engineering and measurement, 2008, pp. 288–290.
P. Runeson, ―Using students as experiment subjects–an analysis on graduate and freshmen student data,‖ Proc. 7th Int. Conf. Empir. Assess. Eval. Softw. Eng., pp. 95–102, 2003.

Plan for continuing the work beyond the ISERN session: Collaboration on future studies that allow to compare types of experimental subjects.

Naming the Pain in Requirements Engineering

Chairs: Daniel Méndez Fernández & Stefan Wagner

Abstract: During last year's ISERN meeting, we conducted a thematic workshop to initiate a globally distributed family of surveys on requirements engineering, called NaPiRE (Naming the Pain in Requirements Engineering) hosted under the umbrella of ISERN. We presented the design, the first results we could already obtain from Germany, and we openly discussed how to potentially proceed by forming a long-term research collaboration where different researchers conduct an independent replication in their respective (affiliation) country. This included the initial re-adjustment of the instrument for the first replication, team building for conducting the survey in different countries and the establishment of a process. In the meantime, we could form a team of 18 researchers and we are setting up the infrastructure to conduct the survey in 12 countries. Details about the project can be taken from http://www.re-survey.org
As part of this upcoming thematic workshop, we plan to discuss the lessons learnt and prepare the synthesis of our results to organise the subsequent dissemination as a joint publication. The long-term vision is to repeat the procedure every two years where we re-adjust the used instrument in one year before conducting the survey and disseminating the results in the following.

Session goals: We plan to use the open space event during day 1 to jointly discuss the current results from the surveys performed in 12 countries as a preparation of the workshop.
The goal of the thematic workshop is to jointly
- Discuss the lessons learnt from conducting the first replication round as an input for the workshop in 2015
- Initiate the synthesis of the results (to be continued and published after the ISERN session)
- Bring in new researchers in participating in future replications
The outcome of the workshop shall be a collection of improvements to the instrument and replication process, a plan for the synthesis and a list of people interested in the next replications.

Development of the session: To conduct the workshop in a systematic manner, we propose to:
1. Introduce the complete NaPiRE concept and report on the results so far
2. Collect and discuss lessons learned (or feedback on the instrument respectively) in group work
3. Form synthesis process and time plan in open discussion

Background and recommended reading list: Members who already conducted the RE survey in this year are expected to participate. Members not previously involved should read the information provided on the website http://www.re-survey.org as well as the following document that describes the full design, the results and instrument of the survey of 2013 (for internal use only): http://www4.in.tum.de/~mendezfe/openspace/NaPiRE/INFSOF-S-13-00428.pdf

Plan for continuing the work beyond the ISERN session: The whole workshop is intended to strengthen the collaboration and plan the dissemination of the results after the ISERN session. We will continue to steer the family of surveys over the next years and to build a continuum in this project under the umbrella of ISERN whereby this workshop builds one consequent next step in this endeavour.

Technical Debt

Chairs: Davide Falessi, Carolyn Seaman

Abstract: Technical Debt refers to a phenomenon that often occurs in software maintenance, in particular of large legacy systems, in which decisions are made that result in short-term gain (e.g. increased productivity, shorter cycle time) at the expense of a more uncertain long-term cost (e.g. increased complexity, more fragile code, more defects, deferred work). In particular, Technical Debt most commonly refers to the cumulative effect of such decisions over time. Such decisions represent a type of “debt” that the development team “owes” the system under maintenance, in the form of more time that should have been spent. This “debt” will eventually incur “interest” in the form of more expensive maintenance tasks in the future. However, unlike the analogous financial debt, the timing, amount, and form of the interest is uncertain, which makes the management of technical debt especially challenging. Specific examples of Technical Debt range from code smells (resulting from rushed modifications that violate good programming practice), to modularity violations (resulting from rushed modifications that violate architectural principles), to large backlogs of minor defects (resulting from deferring low-priority defect fixes), to inadequate documentation (resulting from deferring documentation writing and updating), to very expensive infrastructure upgrades (resulting from delaying the upgrade).
The Technical Debt metaphor is in many ways a restatement of a general software engineering problem that has existed as long as software has. It is not a new idea, but a new way of talking about and reasoning about software maintenance problems. Many areas of empirical software engineering research potentially could contribute to the Technical Debt research agenda, even though the researchers conducting it do not use this terminology. Long-term streams of research in program analysis, software metrics, project management, software maintenance and evolution, and risk management have all contributed to Technical Debt research, and will continue to do so. "

Session goals: The goal of this workshop is to identify areas of research in which ISERN members are already engaged that could contribute to the Technical Debt research agenda, and thus identify potential new collaborations by interpreting existing research goals and agendas using this metaphor.

Development of the session: - Opening presentation
- the Technical Debt metaphor
- the contribution of various research streams
- current Technical Debt research
- open research questions
- Participant introductions
- what each of us is working on
- how could our current research contribute to the Technical Debt research agenda?
- Break into small groups to discuss more specific topics and potential collaborations
- Report back and document specific collaborations to follow up on

Background and recommended reading list:

Plan for continuing the work beyond the ISERN session: - Workshop organizers will check in with each of the identified collaborator groups 2-3 times over the following year to encourage collaboration and determine progress. We will report back at ISERN 2015.

Creating an ISERN Community Experience Base

Chairs: Victor Basili, Jens Heidrich, Natalia Juristo, Forrest Shull and Laurie Williams

Abstract: It is important that the Empirical Software Engineering community make it clear to the larger community, e.g., grant givers, companies, fellow researchers, what we have learned from our numerous studies and the impact it has had on the community. One of our problems has been our inability to share and combine our results and begin to identify the context variable that cause variations in results. We need to create a community experience base (CEB) which combines studies, exposes important context variables, shows the impact of these studies on software engineering knowledge and practice. This is not an easy task but should be the role of ISERN.

Session goals: Understand what is needed to build such a CEB and asks if we can and should create such a base

Building a community experience base is not a simple task and there are lots of issues that need to be dealt with

What should go into it? (how are context variables represented)

Where do we keep it and how do we maintain it?

How do we contribute?

What are the rewards for contributing? (Academics specifically need rewards, in the form of papers and the like, e.g., physicists typical have short papers dominated by author lists – Maybe we can have a new mechanism for contributors and users of the CEB )

What are the criteria for accepting a contribution, e.g., multiple studies?

How do we define 'learned' and 'impact'?

etc.

Development of the session: Chairs will start off with questions, options, and topic areas. Members of the audience will participate in discussions. Notes will be taken and provided back to the ISERN members.

Background and recommended reading list:

Plan for continuing the work beyond the ISERN session: This should be a standing activity of ISERN and is open for various ISERN members to actively, e.g., serve on a CEB board

Psychoempirical Software Engineering

Chairs: Daniel Graziotin, Pekka Abrahamsson, Andreas Jedlitschka

Abstract: Software is developed by humans, and it is employed by humans. Practitioners know this well, and claim that the way to get the best from developers is to focus on them as people. This claim, also echoed by Boehm more than 25 years ago, is appealing but very difficult to verify in research. The software construction process is mainly intellectual. Recently, the discipline of software engineering has begun to adopt a multidisciplinary view and has embraced theories from more established disciplines. However, Empirical software engineering (ESE) has too often ignored psychological measurements with perhaps the exception of personality tests. There is an opportunity for ESE to quantitatively studying human aspects. For example, research in organizational psychology suggests that studying affects (emotions and moods) on the job is fundamental for understanding performance issues currently unexplained by ESE research. Affects are deeply linked to intellectual performance and cognitive activities, especially with analytical problem solving and creativity. Recent ESE research is determining how deep is the linkage between affect and software development, and the first results are fairly positive. This, together with past studies on commitment in software process improvement, encourages the workshop chairs to call for an agenda of Psychoempirical Software Engineering, and to foster collaboration among ISERN members.
Our aim at the workshop is to report the results of our employment of psychological measurements in ESE research, to illustrate the theory behind affects, the different ways to measure them, the different ways to measure the creativity and the problem-solving performance of software developers, and to initiate research collaboration with ISERN members. We would like to collect the ISERN members’ experience with psychological measurements in ESE, and to gather desirable human aspects to be collaboratively studied in the near future.

Session goals:
1. Present the motivation for incorporating psychological measurements in ESE.
2. Offer our experience in using measurements of affects and problem solving performance in ESE, including theory and measurement tools.
3. Exchange experience and lessons learned with ISERN members.
4. Agree on research collaboration with ISERN members and an agenda for Psychoempirical Software Engineering.

Development of the session:

Presentation of the motivation for incorporating psychological measurements in ESE. A focus during the workshop will be provided about the measurements of affect (emotions and moods), as it is in our background. Analytical Problem-solving and creativity assessment will be included as well.

Psychological measurements. Where to find them, how to choose them, how to administer them, how to make sense of the results, their reliability, and the drawbacks in employing them.

We will briefly present two studies, which employed affect measurements and measurements of performance in terms of analytical problem solving, creativity, and the self-assessed productivity.

Depending on the number of participants, a discussion-enhancing “game” is proposed. The “game’s” rules change slightly according to the number of participants. However, the essence of the “game” is the following: participants mention human aspects that they studied in the past, and human aspects that they think need further exploration. The discussion--clustered into topics and run simultaneously if the number of participants is high--is then fostered according to the results.

Either during the game (if played) or during the workshop, an introduction on affects (emotions and moods) will be given.
o Theory that affects are likely going to replace other constructs, e.g., commitment, motivation, and job satisfaction.
o The relationship between affects and job-related performance, in particular on intellectual activities like software development.

The game itself will facilitate the exchange of lessons learned with ISERN members.

The results from the game will make potential for collaboration among ISERN members transparent.

Bonus: setup of a Psychoempirical Software Engineering agenda

Background and recommended reading list: Palacios, R. (2010). A study of emotions in requirements engineering. In Organizational, Business, and Technological Aspects of the Knowledge Society (pp. 1–7). doi:10.1007/978-3-642-16324-1_1
Denning, P. J. (2012). Moods. Communications of the ACM, 55(12), 33. doi:10.1145/2380656.2380668
Feldt, R., Torkar, R., Angelis, L., & Samuelsson, M. (2008). Towards individualized software engineering. In Proceedings of the 2008 international workshop on Cooperative and human aspects of software
engineering - CHASE ’08 (pp. 49–52). New York, New York, USA: ACM Press. doi:10.1145/1370114.1370127
Graziotin, D., Wang, X., & Abrahamsson, P. (2013). Are Happy Developers More Productive? The Correlation of Affective States of Software Developers and Their Self-assessed Productivity. In J. Heidrich, M. Oivo, A. Jedlitschka, & M. T. Baldassarre (Eds.), 14th International Conference on Product-Focused Software Process Improvement (PROFES 2013) (pp. 50–64). Paphos, Cyprus: Springer Berlin Heidelberg. doi:10.1007/978-3-642-39259-7_7
Graziotin, D., Wang, X., & Abrahamsson, P. (2014). Happy software developers solve problems better: psychological measurements in empirical software engineering. PeerJ, 2, e289. doi:10.7717/peerj.289
Khan, I. A., Brinkman, W., & Hierons, R. M. (2010). Do moods affect programmers’ debug performance? Cognition, Technology & Work, 13(4), 245–258. doi:10.1007/s10111-010-0164-1

Plan for continuing the work beyond the ISERN session: The outcome of the workshop shall be an informal agreement on the collaboration and its operationalization. A desirable outcome would be to start the setting up of an agenda for Psychoempirical Software Engineering.

Collaboration on future studies about human aspects using psychological measurements. The session chairs will drive that effort and will be happy to invite interested contributors.

Agenda for Psychoempirical Software Engineering.

Workshop Summary

Chairs: Mike Barker, Barbara Russo

Each group describes conclusions, outcomes and future actions in 10 minutes

ISERN Business

Chairs: Dieter Rombach