Can computers really mark exams? Benefits of ELT automated assessments

ɫèAV Languages
Hands typing at a laptop with symbols

Automated assessment, including the use of Artificial Intelligence (AI), is one of the latest education tech solutions. It speeds up exam marking times, removes human biases, and is as accurate and at least as reliable as human examiners. As innovations go, this one is a real game-changer for teachers and students. 

However, it has understandably been met with many questions and sometimes skepticism in the ELT community – can computers really mark speaking and writing exams accurately? 

The answer is a resounding yes. Students from all parts of the world already take AI-graded tests.  and Versanttests – for example – provide unbiased, fair and fast automated scoring for speaking and writing exams – irrespective of where the test takers live, or what their accent or gender is. 

This article will explain the main processes involved in AI automated scoring and make the point that AI technologies are built on the foundations of consistent expert human judgments. So, let’s clear up the confusion around automated scoring and AI and look into how it can help teachers and students alike. 

AI versus traditional automated scoring

First of all, let’s distinguish between traditional automated scoring and AI. When we talk about automated scoring, generally, we mean scoring items that are either multiple-choice or cloze items. You may have to reorder sentences, choose from a drop-down list, insert a missing word- that sort of thing. These question types are designed to test particular skills and automated scoring ensures that they can be marked quickly and accurately every time.

While automatically scored items like these can be used to assess receptive skills such as listening and reading comprehension, they cannot mark the productive skills of writing and speaking. Every student's response in writing and speaking items will be different, so how can computers mark them?

This is where AI comes in. 

We hear a lot about how AI is increasingly being used in areas where there is a need to deal with large amounts of unstructured data, effectively and 100% accurately – like in medical diagnostics, for example. In language testing, AI uses specialized computer software to grade written and oral tests. 

How AI is used to score speaking exams

The first step is to build an acoustic model for each language that can recognize speech and convert it into waveforms and text. While this technology used to be very unusual, most of our smartphones can do this now. 

These acoustic models are then trained to score every single prompt or item on a test. We do this by using human expert raters to score the items first, using double marking. They score hundreds of oral responses for each item, and these ‘Standards’ are then used to train the engine. 

Next, we validate the trained engine by feeding in many more human-marked items, and check that the machine scores are very highly correlated to the human scores. If this doesn’t happen for any item, we remove it, as it must match the standard set by human markers. We expect a correlation of between .95-.99. That means that tests will be marked between 95-99% exactly the same as human-marked samples. 

This is incredibly high compared to the reliability of human-marked speaking tests. In essence, we use a group of highly expert human raters to train the AI engine, and then their standard is replicated time after time.  

How AI is used to score writing exams

Our AI writing scoring uses a technology called . LSA is a natural language processing technique that can analyze and score writing, based on the meaning behind words – and not just their superficial characteristics. 

Similarly to our speech recognition acoustic models, we first establish a language-specific text recognition model. We feed a large amount of text into the system, and LSA uses artificial intelligence to learn the patterns of how words relate to each other and are used in, for example, the English language. 

Once the language model has been established, we train the engine to score every written item on a test. As in speaking items, we do this by using human expert raters to score the items first, using double marking. They score many hundreds of written responses for each item, and these ‘Standards’ are then used to train the engine. We then validate the trained engine by feeding in many more human-marked items, and check that the machine scores are very highly correlated to the human scores. 

The benchmark is always the expert human scores. If our AI system doesn’t closely match the scores given by human markers, we remove the item, as it is essential to match the standard set by human markers.

AI’s ability to mark multiple traits 

One of the challenges human markers face in scoring speaking and written items is assessing many traits on a single item. For example, when assessing and scoring speaking, they may need to give separate scores for content, fluency and pronunciation. 

In written responses, markers may need to score a piece of writing for vocabulary, style and grammar. Effectively, they may need to mark every single item at least three times, maybe more. However, once we have trained the AI systems on every trait score in speaking and writing, they can then mark items on any number of traits instantaneously – and without error. 

AI’s lack of bias

A fundamental premise for any test is that no advantage or disadvantage should be given to any candidate. In other words, there should be no positive or negative bias. This can be very difficult to achieve in human-marked speaking and written assessments. In fact, candidates often feel they may have received a different score if someone else had heard them or read their work.

Our AI systems eradicate the issue of bias. This is done by ensuring our speaking and writing AI systems are trained on an extensive range of human accents and writing types. 

We don’t want perfect native-speaking accents or writing styles to train our engines. We use representative non-native samples from across the world. When we initially set up our AI systems for speaking and writing scoring, we trialed our items and trained our engines using millions of student responses. We continue to do this now as new items are developed.

The benefits of AI automated assessment

There is nothing wrong with hand-marking homework tests and exams. In fact, it is essential for teachers to get to know their students and provide personal feedback and advice. However, manually correcting hundreds of tests, daily or weekly, can be repetitive, time-consuming, not always reliable and takes time away from working alongside students in the classroom. The use of AI in formative and summative assessments can increase assessed practice time for students and reduce the marking load for teachers.

Language learning takes time, lots of time to progress to high levels of proficiency. The blended use of AI can:

  • address the increasing importance of formative assessmentto drive personalized learning and diagnostic assessment feedback 

  • allow students to practice and get instant feedback inside and outside of allocated teaching time

  • address the issue of teacher workload

  • create a virtuous combination between humans and machines, taking advantage of what humans do best and what machines do best. 

  • provide fair, fast and unbiased summative assessment scores in high-stakes testing.

We hope this article has answered a few burning questions about how AI is used to assess speaking and writing in our language tests. An interesting quote from Fei-Fei Li, Chief scientist at Google and Stanford Professor describes AI like this:

“I often tell my students not to be misled by the name ‘artificial intelligence’ — there is nothing artificial about it; A.I. is made by humans, intended to behave [like] humans and, ultimately, to impact human lives and human society.”

AI in formative and summative assessments will never replace the role of teachers. AI will support teachers, provide endless opportunities for students to improve, and provide a solution to slow, unreliable and often unfair high-stakes assessments.

Examples of AI assessments in ELT

At ɫèAV, we have developed a range of assessments using AI technology, including , aimed at those who need to prove their level of English for a university place, a job or a visa. It uses AI to score tests and results are available within five days. 

Read more about the use of AI in our learning and testing here.

Explore our courseware and exams

More blogs from ɫèAV

  • A woman/teacher teaching her students in the classroom.

    Top tips for writing a winning English Teacher Awards 2024 nomination

    By Thomas Gardner
    Reading time: 3 minutes

    We are celebrating teaching excellence around the world through the ɫèAV English Teacher Awards 2024.

    Open to all types of English language educator, the 15 lucky global winners will take home thousands of pounds of prizes, from trips to conferences to the latest language learning software for their school.

    Nominating an educator for the ɫèAV English Teacher Awards is simple. All applications are made via the awards entry page, and you have 600-1000 words to tell us how your nominee builds learners’ confidence to be themselves in English.

    Here are our top tips to make the most of every word and ensure your entry is top of the class.

    Application tips

    1. Make it category specific

    One of the core judging criteria is how well an application matches the category description.

    Educators can be nominated in one of five different categories:

    1. Teaching Young Minds English
    2. Empowering Teen Confidence in English
    3. Cultivating Lifelong Learners in English
    4. Innovation in English Language Teaching
    5. Rising Stars of English Language Teaching

    Tailoring your answer to the category description will make sure our judges understand exactly why your nominee is the perfect winner.

    For a detailed description of each of the categories, take a look at our recent blog post.

    2. Focus on confidence

    Learning a language isn’t just about knowing the vocabulary or understanding the grammar; it’s about having the confidence to go out there and use your newfound skills to unlock a world of opportunities.

    But feeling confident in something you’re still learning is hard. Anyone that’s learnt a language has experienced those dips in motivation and confidence when you come across a topic that doesn’t come naturally to you.

    That’s where a great teacher can make all the difference. The right support and encouragement combined with innovative teaching techniques can help learners protect and even build their confidence in challenging moments.

    That confidence is what transforms language learning from just communicating to feeling able to really be yourself in English. And when you’re able to do that, great opportunities come your way.

    Our judges are looking for nominations that understand the connection between confidence and self-expression and the impact that can have on learners’ lives.

    3. Demonstrate your impact

    Including relevant data and examples in your application will help judges understand the impact your nominee has had.

    This might include data such as internal or external test results, as well as how those results have changed during that teacher’s tenure.

    Evidence doesn’t have to mean numbers. Judges also welcome anecdotes and examples that describe the impact an educator has had on learners’ confidence.

    The judging process

    Nominations can be submitted between 15th August and midnight (CST) on 1st November 2024.

    After that date, all entries will be reviewed and a shortlist submitted to our expert judges.

    There are two to three judges for each category who will independently review and shortlist nominees before collectively awarding a Gold, Silver, and Bronze winner in their category.

    Shortlisted entries will be contacted by our team and announced at the ɫèAV Teacher Awards ceremony on November 21st 2024.

    Meet the judges

    From Argentina to Algeria, Italy to Germany, ɫèAV English Teacher Award 2024 judges come from all over the world.

    Many have years of ELT teaching experience, as well as expertise from academia and business.

    Here are just three of our expert judges:

    • Belgin Elmas: Head of the English Language Teaching Department at TED University in Ankara, Belgin is a professor with more than 34 years of teaching experience.
    • Dr Silvia Minardi: President of Lingua e Nuova Didattica and Vice President of Aermigo. Dr Minardi has extensive experience in CLIL training, collaborating with the ECML and the Council of Europe.
    • Dr Renata Condi: An experienced teacher, development professional, and course book author, Dr Condi holds a PhD and MA in Applied Linguistics and an MBA in School Management.

    The prizes

    With a Gold, Silver, and Bronze winner in each of the five categories, there are 15 chances for educators and their schools to win.

    Gold winners will receive a trip to a regional professional conference, including flight and hotel, as well as up to £500 in credit for professional development.

    Their school will win annual licenses to top language learning app, Mondly by ɫèAV as well as a course, assessment, and certification bundle valued at £1000

    Silver and Bronze winners will take home top prizes too, with £250 in credit towards their professional development and up to 500 Mondly by ɫèAV licenses for their school to continue their good work building learners’ confidence in English.

  • Childten sat at a desk staring up at a tablet

    Embracing AI for the new school year

    By Thomas Gardner
    Reading time: 2 minutes

    Back-to-school season is the perfect time to introduce something new into your teaching toolkit. With advancements in technology, Artificial Intelligence (AI) is becoming increasingly important for educators to incorporate into their workflow and beneficial for students. Here are some practical ways AI can enhance your classroom experience this new school year.

    AI saves valuable time

    One of the biggest challenges teachers face is finding the time to plan and create engaging lessons. AI can maximize your time by generating word sets, images and activities. Imagine having a virtual assistant that can suggest fresh ideas for your lessons, create tailored resources and even develop entire lesson plans. By automating these time-consuming tasks, AI allows you to focus more on teaching and interacting with your students.

    How AI can help

    • Lesson planning: Generate detailed lesson plans and activities.
    • Lesson inspiration: Get new and creative ideas for classroom activities.
    • Resource creation: Develop word sets, images and other teaching materials.

    As you prepare for the upcoming school year, consider how integrating AI tools can elevate your teaching practice, making your lessons more engaging while freeing up precious time for interactions with your students.

    Personalized learning and targeted feedback

    Every student learns differently, and tailoring lessons to meet individual needs can be challenging. AI can help by providing personalized learning experiences and targeted feedback. Using data-driven insights, AI tools can identify areas where learners are struggling and offer customized recommendations to help them improve. This not only enhances the learning experience but also ensures that no student is left behind.

    Benefits of AI in personalization

    • Personalized lessons: Adapt lessons to suit individual learning styles.
    • Targeted feedback: Provide specific feedback to help students improve.

    Creating an inclusive classroom

    AI can play an important role in creating an inclusive classroom environment. AI can generate visuals, audio, videos and text, or a combination of all four, catering to different learning preferences. This ensures that all learners, regardless of their learning styles or abilities, have access to the same educational content.

    AI in inclusion

    • Multimodal content: Use various formats to cater to different learning preferences.
    • Accessibility: Ensure that all students can engage with the material.

    Supporting independent learning

    Encouraging students to take charge of their own learning is essential for their development. AI can support independent learning by providing platforms that facilitate self-study at home. These platforms offer a range of resources and tools that help students learn at their own pace, making education more flexible and accessible.Explore how supports independent learning with immersive AI-powered conversations.

    AI tools for self-study

    • Independent learning: Foster a culture of self-directed learning.
    • Self-study resources: Provide tools and platforms for students to use at home.

    Integrating AI into your teaching practice can revolutionize the way you educate and engage with your students. From saving time on lesson planning to creating an inclusive classroom and supporting independent learning, the benefits are significant.

    Want more AI tips and tricks for the classroom? Join us on the page for the next video in our series.

  • A smiley woman and one of her friends/students working in a cafe.

    English Teacher Awards 2024: Understanding the categories

    By Thomas Gardner
    Reading time: 4 minutes

    Teachers shape every aspect of our learning experience, especially when it comes to language learning. Great teachers give learners not only the skills but the confidence to go out in the world, start speaking up and discovering new opportunities.

    We’re celebrating those exceptional educators with the ɫèAV English Teacher Awards 2024.

    With five different categories and a Gold, Silver and Bronze winner in each, there are 15 chances to take home thousands of pounds worth of top prizes for the winning teachers and their schools.

    Find out more about who can enter and the different categories in this article.