How to Assess AI Literacy Through Problem-Based Learning

Bridging AI Literacy and Classroom Practice

As educational leaders increasingly reference frameworks like the UNESCO AI Competency Framework, the USC "Understand–Evaluate–Use" model, and the Barnard College AI Literacy Framework, the challenge shifts from definition to implementation. How do we move beyond theoretical knowledge to confirm that students can actually apply these concepts in a real-world context? By integrating these frameworks into problem-based learning (PBL), educators can transform abstract competency goals into concrete, assessable moments.

For many teachers, the gap lies in the transition between understanding how AI works and being able to evaluate its output. Whether you are aligning with the Stanford Teaching Commons guidance on AI literacy or international benchmarks like the World Bank SABER-ICT and OECD PISA ICT Familiarity, the most effective way to track progress is through active, iterative formative assessment.

At a Glance

• Best for: School teachers and curriculum leads.
• The problem: Converting complex AI literacy frameworks into actionable student assessments.
• Saval capability used: AI-powered formative assessment and gamified leaderboards.
• Plan needed: Free plan for basic assessment; Pro plan for advanced analytics and gamification.
• Time to try it: About 15 minutes to configure a challenge scenario.

AI literacy is the ability to critically analyze algorithmic outputs and ethically apply generative tools. Teachers can assess this by presenting students with real-world, problem-based scenarios where they must evaluate an AI's response for accuracy, bias, and logic. Using a tool like saval.app, educators can gamify this evaluation process, providing instant feedback that helps students refine their analytical skills in real-time.

How to Build an AI Literacy Challenge with Saval

You can turn framework-aligned objectives into a live gamified experience by following this workflow:

1. Identify the framework objective: Choose a specific competency, such as "Evaluating AI-generated text for bias," based on the USC "Understand–Evaluate–Use" framework.
2. Create a Problem-Based Scenario: In Saval, draft a challenge where students must review an AI-generated essay or code snippet. Use the Problem-Based Scenarios feature to embed the reference text.
3. Set the Rubric: Define your grading criteria so the AI can provide consistent feedback on the student's evaluation of the prompt. You can configure AI strictness to match your pedagogical goals.
4. Launch the Game: Use the Class Quiz mode to start the session. Students join via a QR code, fostering immediate engagement.
5. Monitor Progress: Use the Performance Dashboard (Essential plan) to see which students struggle with the "Evaluate" aspect of the framework.

What This Means for Your Role

For the classroom teacher, this approach shifts the focus from managing technology to facilitating critical thinking. You maintain control through custom rubrics and the ability to hide or show AI feedback, ensuring that the technology remains a tool for your instructional design. While Saval manages the grading and engagement metrics, the teacher remains the final authority on the qualitative depth of student responses.

FAQ

Can I use these frameworks to build quizzes? Yes, you can map specific learning objectives from the UNESCO or Barnard frameworks directly to individual questions within a Saval challenge.

How does gamification help with literacy? Gamification, such as the Fly or Fall mode (Pro plan), keeps students engaged during iterative feedback cycles, which is essential when teaching complex, nuanced topics like AI ethics.

Takeaways

• Use existing AI literacy frameworks to design specific "Evaluate" tasks rather than generic tests.
• Leverage gamification to turn formative assessment into a low-stakes, high-engagement activity.
• Use Learning Analytics to identify which AI literacy competencies require more explicit instruction.
• Start with a simple Class Quiz to test student baseline knowledge before moving to complex problem-based scenarios.

New teacher accounts include a 30-day Enterprise trial, so you can test AI marking and analytics with a real class before choosing a plan. Sign up to get started.