AEA-Europe 2024

Advances in Educational Assessment Practices:

Considering the use of Technology, Artificial Intelligence, and Process Data for Assessment in the 21^st Century

Educational assessment has long been an integral part of educational systems worldwide, guiding our understanding of student learning and informing educational policies and practices. Within these systems, paper-and-pencil tests have traditionally been the primary method for assessing students’ knowledge and skills. In the 21^st century however, with the advancement of technology, the wider adoption of computer-based tests, and the availability of vast amounts of process data, the landscape of educational assessment is evolving very rapidly. Moreover, the recent emergence and wide availability of generative artificial intelligence (AI) models such as ChatGPT is currently pushing the field into more innovative directions.  Recognizing the potential of integrating AI and process data for transforming the way we evaluate, deliver feedback, and understand student learning, we are provided with research opportunities and valuable insights that go beyond the mere quantification of correct and incorrect answers from traditional paper-and-pencil tests.

The first focal point of the conference theme is Artificial Intelligence and Assessment, which promises to revolutionize the assessment landscape. Machine learning algorithms offer the potential to enhance the validity, reliability, and efficiency of testing procedures. AI-driven assessment systems can adapt to individual students’ learning profiles, identifying strengths and areas for improvement to provide personalized feedback and learning pathways. As we explore the application of AI in assessment, we must also address issues related to the misuse of AI, as well as issues of ethics, fairness, transparency, accessibility, and bias to ensure that the implementation aligns with ethical considerations.

The second focal point of the conference theme is Leveraging Process Data in Assessment, by delving into the richness of data that emerge from students’ interaction processes with a test. The analysis of process data, encompassing response times, answer changes, keystrokes, and other behavioral indicators, can provide unprecedented opportunities to gain deep insights into students’ thought processes, learning strategies, and test-taking strategies. By understanding the examinee cognitive processes during test-taking, we can tailor interventions and instructional strategies to meet their specific needs, ultimately fostering more effective learning outcomes.

The Relevance of Traditional Testing is the third focal point of this conference theme, where we challenge the confines of conventional assessment methods. As technology advances, so does the potential for diverse forms of assessment, including, but not limited to, computerized adaptive testing, game-based assessments, authentic assessment and new approaches to the assessment of 21^st century skills. Some of these approaches can capitalize on process data while also enabling a better understanding of students’ competencies, problem-solving skills, and abilities to apply knowledge in real-world contexts. However, existing approaches to assessment are supported by a long tradition of research, custom and practice, factors that suggest continued relevance for such methods.

Students, educators, and institutions all play essential roles in the implementation of assessment practices. Assessment Perspectives and Experiences of Stakeholders are of paramount importance in understanding the perception and implementation of assessment within formal educational and in extracurricular contexts. Inquiry into how these viewpoints influence individuals’ behaviors and learning outcomes will uncover valuable implications, test score interpretations and validity evidence for assessments.  The exploration of various methodologies and practices (e.g., test development or scoring with AI technology, innovative item forms and delivery platforms, and more holistic approaches to assessment) across diverse settings, academic subjects, and administration conditions will reveal promising and innovative uses of assessments.

Finally, as we embrace the power of process data and artificial intelligence, it is essential to address the Ethical and Practical Considerations associated with assessment practices. We must engage in meaningful discussions on the ethical implications of all assessment including the use of AI in assessment, ensuring that student privacy, data security, and equity remain at the forefront of our endeavors. Additionally, identifying best practices for implementing AI-driven assessment is crucial to harnessing its potential for the benefit of all learners, and of education in general.

The conference theme for 2024 builds on the theme of the successful 2023 conference titled “Assessment reform journeys: intentions, enactment and evaluation”. By building on the momentum of the previous conference, we are excited to continue the exploration by investigating and improving traditional assessment practices and drawing attention to alternative assessment strategies that utilize process data and AI models, amongst others. In light of this theme, we extend a warm invitation to scholars, educators, researchers, students, and policymakers to share their innovative research and transformative ideas in these and related areas.

For the Annual Conference of the Association for Educational Assessment- Europe 2024, in Paphos, Cyprus, we encourage submissions that delve into various aspects of assessment. Possible topics for the conference can include but are not limited to:

• Innovative forms of assessment
• Process data in assessment
• Revolutionizing assessment, including through AI models
• Ethical issues in assessment, including in the use of AI and process data in assessment
• Assessing non-cognitive skills
• The potential of authentic and game-based assessment in the 21^st century
• Assessment feedback
• Achieving equity and inclusion in diverse assessment occasions
• Validity issues in assessment, including in large-scale assessment
• Relevance of traditional approaches to assessment
• Examinee test-taking strategies
• Assessment adaptations for examinees with learning disabilities
• Assessment perspectives, cultures, and experiences