You are here
Assessments - Computer Science
Developing and validating reliable assessment instruments is a long process involving investigation of student difficulties, designing questions which can reliably uncover these difficulties, interviewing faculty about the appropriateness of the questions, pilot testing with students (both individual interviews and large scale in-class testing), performing statistical analysis, question refinement (addition/change/removal) and re-testing. It takes years of development effort to create and validate reliable assessment instruments and in order to ensure that these assessment instruments do not lose their reliability (for example, by answers showing up in online forums) it is important that:
1. students are not given copies following administration of the assessment tool and
2. questions are not incorporated into web-based question delivery systems without adequate security to prevent printing or unauthorized access by students.
For a list of important and difficult concepts in introductory computer science courses, see Goldman K. et al, Identifying Important and Difficult Concepts in Introductory Computing Courses using a Delphi Process, Proceedings of the 39th SIGCSE technical symposium on Computer science education, 256-260 (2008).
In the paper below, researchers have used a Delphi process to identify the core concepts and topics which are important and difficult in three introductory computer science courses: discrete mathematics, programming fundamentals and logic design. The topic rankings described in the paper can be useful in developing concept inventories for these courses.
Content-based Assessments
- Foundational CS1 (FCS1)
- 27-item multiple choice conceptual assessment of foundational computer science concepts. This inventory is independent of programming language used in a CS1 course.
- To obtain the instrument, contact Allison Elliot Tew at aetew@cs.ubc.ca or Mark Guzidal at guzidal@cc.gatech.edu.
- Detailed analysis and evaluation results for the instrument is included in this dissertation: A. E. Tew, Assessing fundamental introductory computing concepts in a language independent manner.
- See also: Parker, M. C., Guzdial, M., & Engleman, S. (2016, August). Replication, validation, and use of a language independent CS1 knowledge assessment. In Proceedings of the 2016 ACM conference on international computing education research (pp. 93-101).
- Computer Engineering Concept Inventory (CPECI)
- Concept inventory of conceptual understanding of computer engineering appropriate for students who have completed sophomore level courses, such as Digital Logic, Computer Design, Data Structures or Introduction to Computer Organization.
- Concept Inventory for Basic Data Structures
- Designed to be used in conjunction with CS2 courses that teach basic data structures
- Data structures included on the instrument are Array-Based Lists, Linked-Lists, and Binary Search Trees
- Available by joining this google group
- Digital Logic Concept Inventory (DLCI)
- 24 item multiple choice assessment of digital logic concepts.
- Inventory is printed in Geoffrey Herman’s dissertation: G. L. Herman, The development of a digital logic concept inventory
- Discrete Mathematics Concept Inventory (DMCI)
- Work under progress to develop an inventory for Discrete Mathematics.
- For more information, contact Vicki Almstrum at almstrum@cs.utexas.edu.
- Reasoning Concept Inventory for Computer Science
- Inventory of basic reasoning principles for computer science.
- Operating Systems Concept Inventory
- 10 item multiple choice instrument to measure student understanding of concepts covered in an operating systems course.
- Authors intend to establish the validity and reliability of the instrument in the near future.
- Available here.
- Computer Architecture Assessment
- These researchers developed a multiple choice assessment of computer architecture concepts. However, they have not yet subjected the assessment to rigorous statistical tests (e.g., reliability).
- Algorithms and Data Structures Concept Inventory
- Work in progress by researchers at University of Dortmund to develop a concept inventory for Algorithms and Data Structures courses by identifying student misconceptions and developing multiple choice items with strong distractors.
- For more information, contact Holger Danielsiek at holger.danielsiek@udo.edu.
- Concept Inventory questions for student misconceptions regarding binary trees and hash tables
- Recursion: Hamouda, S., Edwards, S. H., Elmongui, H. G., Ernst, J. V., & Shaffer, C. A. (2017). A basic recursion concept inventory. Computer Science Education, 27(2), 121-148.
Affect and Attitudes about Computer Science
- Computer Science Attitude Survey
- Instrument which can be used to assess student attitudes toward computer programming and computer science in general on five subscales: Confidence in learning computer science and programming, Attitude toward success in computer science, Computer Science as a male domain, Usefulness of computer science and programming, Effective motivation in computer science and programming
Computer Science and Programming Self-efficacy
- Computer Science Self-Efficacy Scale for CS0 Courses
- Instrument designed to measure perceptions of capability associated with success in CS0 students
- Computer Programming Self-Efficacy Scale
- 32 item instrument designed to assess students’ self-efficacy about programming appropriate for an introductory computer science course
- Article describing its development and validation available here.
If we missed some assessments developed by computer science education researchers, please let us know at dbserc@pitt.edu.