Using Computer Vision to Assess Lab Skills

The integration of artificial intelligence into educational settings is rapidly transforming traditional approaches to skill assessment. Among these advancements, computer vision stands out as a particularly promising tool, providing educators with novel opportunities to observe and evaluate student performance in laboratory environments. This article explores the practicalities, educational value, and ethical considerations of using computer vision—specifically through a setup involving a webcam and Teachable Machine—to assess laboratory skills.

Understanding Computer Vision in Educational Contexts

Computer vision encompasses a range of algorithms and models that enable computers to interpret and process visual information from the world. In recent years, the accessibility of these technologies has grown, making them available for use even in resource-constrained educational environments. For teachers and laboratory coordinators, this means the possibility of tracking students’ activities and providing feedback in ways that were previously unfeasible.

“Computer vision brings a level of objectivity and scalability to skill assessment that manual observation cannot match.”

By analyzing video feeds, computer vision systems can detect gestures, recognize objects, and even classify complex actions. This capability allows educators to monitor how students handle lab equipment, follow safety protocols, and perform specific procedures, all in real time.

The Webcam & Teachable Machine Setup

A practical and approachable way for educators to experiment with computer vision is by using a standard webcam in combination with Teachable Machine, a free web-based tool developed by Google. Teachable Machine enables users to create custom machine learning models without the need for programming expertise.

The basic workflow involves:

• Setting up a webcam to capture students’ actions at the lab bench.
• Collecting example video clips or images of specific skills or procedures.
• Training a model in Teachable Machine to recognize these actions or gestures.
• Deploying the trained model to give instant feedback or to log performance data for later review.

For example, a chemistry instructor might record students demonstrating correct pipetting technique, safe handling of reagents, or accurate use of a microscope. The model can then be trained to distinguish between correct and incorrect methods, flagging errors or confirming proficiency as students work.

Benefits for Educators and Learners

The use of such a setup offers several important advantages:

• Scalability: One teacher can monitor multiple students simultaneously, freeing time for more individualized instruction where needed.
• Objectivity: Automated assessment reduces the influence of subjective bias, providing fairer and more reliable feedback.
• Immediate feedback: Students can receive real-time guidance, helping them correct mistakes and internalize best practices more rapidly.
• Data-driven insights: Aggregate data on student performance enables targeted curriculum improvements and identifies areas where additional support is needed.

*The value of such feedback is not limited to students; teachers can also use the data to reflect on their teaching strategies and to identify systemic issues in lab instruction.*

Setting Up Your Computer Vision Lab Assessment

To begin implementing computer vision in your laboratory, only modest resources are required:

1. Hardware: A laptop or desktop computer and a standard webcam. Most modern devices are sufficient, and USB webcams are readily available if needed.
2. Software: Access to the internet and the Teachable Machine website. No installation is necessary.
3. Data: Examples of both correct and incorrect skill demonstrations, captured as short video clips or still images.

The process is highly iterative; after initial training, the model’s predictions should be tested and refined with additional data to improve accuracy. This collaborative process can even become part of the learning experience, as students help to label data and discuss what constitutes a good or poor technique.

Designing a Skill Assessment Protocol

To maximize educational value, it is important to design assessment protocols that align with learning objectives. For instance:

• Define a set of core competencies (e.g., assembling glassware, measuring liquids, disposing of waste properly).
• Record several examples of each skill, performed both correctly and incorrectly, to serve as training data.
• Train the model in Teachable Machine, using clear labels (e.g., “correct pipetting,” “incorrect pipetting”).
• Deploy the model during lab sessions, using the webcam to assess students’ actions and provide feedback.

An optional extension is to export the trained model and integrate it into custom educational applications using TensorFlow.js. This enables more advanced features such as automated logging, personalized feedback dashboards, or integration with learning management systems.

Supporting Teachers with Limited Technical Background

One of the most attractive aspects of this approach is its accessibility. Teachable Machine requires no coding skills, and the entire process can be managed through an intuitive graphical interface. This empowers teachers who may not have a technical background to experiment with AI-driven assessment, fostering a culture of innovation and continuous improvement.

“The goal is not to replace human educators, but to augment their ability to observe, assess, and support students.”

By lowering technical barriers, more teachers can participate in the development and evaluation of AI tools, ensuring that they address real classroom needs.

Ethical Considerations and Safeguarding

While the benefits of computer vision in lab assessment are clear, the ethical challenges must be addressed with equal seriousness. The use of video data—particularly footage of students—raises important questions of privacy, consent, and data protection.

European educators must comply with legal frameworks such as the General Data Protection Regulation (GDPR). This includes obtaining informed consent from students (or their guardians, where applicable), transparently communicating the purposes of data collection, and ensuring that video recordings are securely stored and processed.

Best practices include:

• Clearly informing students about what data is collected, how it will be used, and who will have access to it.
• Using the minimum amount of data necessary for the task, and deleting it when no longer needed.
• Ensuring that video feeds are not shared outside the educational context, and that access is restricted to authorized personnel.
• Allowing students to opt out without penalty, and providing equivalent alternatives for skill assessment where possible.
• Regularly reviewing data security protocols and involving school data protection officers in the design of assessment systems.

*Ethical stewardship is essential to building trust and ensuring that AI serves the interests of learners and educators alike.*

The Role of Bias and Fairness

AI models, including those built with Teachable Machine, are only as good as the data used to train them. If training data is biased—for example, containing only examples from certain groups of students or favoring specific body types—then the resulting assessments may be unfair. It is crucial to include diverse examples in the dataset and to regularly audit model performance across different groups.

Educators should be alert to the risk of over-reliance on automated assessment. Computer vision should supplement, not replace, human judgment. Nuanced skills and contextual factors may not be fully captured by current AI systems, and ongoing teacher oversight remains essential.

Pedagogical Opportunities and Future Directions

The adoption of computer vision for lab skill assessment opens new pedagogical opportunities. Teachers can use these tools to support formative assessment, promote self-reflection, and encourage peer learning. For instance, students might review their own recorded actions, compare them to model examples, and identify areas for improvement.

Collaborative model-building activities—where students and teachers define what “good practice” looks like—can deepen understanding and foster a sense of shared responsibility for learning. Moreover, the data generated by these systems can inform curriculum development, professional development for teachers, and research into effective lab instruction.

As the technology matures, future directions may include:

• Integration with augmented reality (AR) to provide step-by-step guidance during lab work.
• Automated detection of safety violations or risky behaviors.
• Personalized learning pathways based on detailed analysis of student performance over time.
• Collaborative learning environments where students receive feedback from both AI and peers.

The possibilities for enhancing science education through AI are only just beginning to emerge. With careful attention to ethics, inclusivity, and pedagogical goals, computer vision can become a valuable ally in preparing students for the demands of modern laboratory work.

Empowering Educators in the Age of AI

The journey toward AI-enhanced education is best undertaken as a community. European teachers, supported by clear guidance and robust legal frameworks, are well positioned to harness these innovations for the benefit of their students. By engaging with tools such as Teachable Machine, educators can demystify AI, contribute to its responsible development, and ensure that technology remains grounded in the needs and aspirations of learners.

“In the laboratory, as in life, learning thrives when observation is paired with reflection and feedback.”

With a spirit of curiosity and care, educators can help shape a future where artificial intelligence amplifies the best of human teaching, making science education more accessible, engaging, and effective for all.