Workshops -- Monday

This workshop will focus on design evaluation for educators. We also welcome participation by people from industry. Our 1998 workshop resulted in a valuable conceptual model for understanding design evaluation. We now seek to further develop our understanding by using the model to structure advice for educators on how best to evaluate design in a variety of situations. Design evaluation is an important subject both from the perspective of software engineering, and the perspective of pedagogy. Any design has consequences, and designers must learn how to determine good and bad consequences that may flow from design decisions. Educators must lead students through this process and provide critical guidance for the future.

Our work on design evaluation stems from our earlier work exploring and promoting the early emphasis on design as an important step forward in OO education. A 1996 workshop resulted in general agreement about the advantages of the approach, but also identified difficulties concerning evaluation of design. The model we developed in 1998 clarifies the role of evaluation within the overall development process, with summative evaluation useful to implementors who need to assess designs, and formative evaluation useful to designers who wish to improve designs. We also outlined how evaluation must be done with respect to particular values, and we identified several alternatives, then enumerated various design representations, design evaluation mechanisms, and feedback methods.

We have already made an excellent start, but there is much yet to be done. The alternatives for representation, evaluation and feedback all need to be better related to appropriate design values. And while the feedback loop makes formative evaluation attractive, summative evaluation should not be ignored. Better understanding of summative evaluation would lead to better advice for people choosing between designs, and would also help to assess the performance of designers. For educators, it would also address the issue of student assessment. Finally, we need to determine how to test and verify our findings.

Prospective 1999 workshop participants are invited to submit a position paper with a contribution on this topic. The workshop will consist of presentations of these papers in the morning session, followed by an afternoon session of discussion and collaboration. As before, we will prepare a report summarizing the workshop and results for the OOPSLA addendum, and make workshop papers and related resources available via our web repository. We will again present an immediate report to the OOPSLA Educators Symposium.