Assessing Undergraduate Digital Forensic Competencies Through Incident Investigation Assignments: Evidence from Log-Based Forensic Analysis

Abstract

Assessing complex investigative competence in a way that is both authentic and reliable remains an open measurement problem in forensic and cybersecurity education, and not only in resource-constrained settings. This paper reports a single-cohort pilot of a log-based incident-investigation assignment, built from multiple case variants, that was used to explore this problem in an undergraduate Computer Forensics course. Ten student groups each received their own synthetic evidence package describing a different multi-stage intrusion, with one or two red herrings planted in each case. Each group submitted a forensic report, which was scored with a six-dimension competency rubric aligned to NIST SP 800-86, the NICE Cybersecurity Workforce Framework, and Bloom’s revised taxonomy. The analytic sample consists of the ten group submissions (about 40 students were enrolled). Since each case variant was completed by only one group, case difficulty and group ability are confounded by design, so case-level patterns are treated as preliminary and hypothesis-generating, not as proof that some scenarios are harder than others. With this caveat in mind, the groups performed strongly and consistently on evidence identification, multi-source correlation, and analytical reasoning. Professional reporting was the weakest dimension. In four of the ten groups, the core analysis was clearly stronger than the report that communicated it, a pattern we call the synthesis-and-reporting gap. Critical evaluation was high on average but bimodal, as two groups missed their red herring entirely. We place the instrument within Kane’s argument-based validity framework, present it as a pilot assessment design rather than a validated measurement instrument, and describe the multi-rater, multi-cohort work needed before stronger claims about validity or reliability can be made.