A systematic approach to evaluating architectural portfolios: architectural portfolio evaluation rubric (APER)

Abstract

This study presents the development and validation process of an architectural portfolio evaluation rubric (APER) aimed at assessing the quality and effectiveness of architectural portfolios. The research adopts a systematic approach, employing a series of consecutive methods to construct and validate the rubric. Firstly, a thorough examination of relevant literature is conducted to determine the appropriate criteria. After, the modified Delphi technique is employed to validate the rubric with input from subject-matter experts. Fifteen field experts attempt to achieve consensus by refining the rubric via multiple rounds of qualitative and quantitative data collection. The study also addresses the raters' agreement and consistency of the rubric through statistical analyses, including Fleiss's kappa coefficient, intra-class correlation coefficient (ICC), and generalizability coefficient based on generalizability theory (G theory). Findings indicate that the APER provides a robust framework for evaluating architectural portfolios, facilitating informed decision-making in academic and professional contexts. The study contributes to the continuing discussion about portfolio assessment approaches in architecture, providing significant insights for learners, practitioners, and scholars.