Explaining computerized English testing in plain English

ɫèAV Languages
a pair of hands typing at a laptop

Research has shown that automated scoring can give more reliable and objective results than human examiners when evaluating a person’s mastery of English. This is because an automated scoring system is impartial, unlike humans, who can be influenced by irrelevant factors such as a test taker’s appearance or body language. Additionally, automated scoring treats regional accents equally, unlike human examiners who may favor accents they are more familiar with. Automated scoring also allows individual features of a spoken or written test question response to be analyzed independent of one another, so that a weakness in one area of language does not affect the scoring of other areas.

was created in response to the demand for a more accurate, objective, secure and relevant test of English. Our automated scoring system is a central feature of the test, and vital to ensuring the delivery of accurate, objective and relevant results – no matter who the test-taker is or where the test is taken.

Development and validation of the scoring system to ensure accuracy

PTE Academic’s automated scoring system was developed after extensive research and field testing. A prototype test was developed and administered to a sample of more than 10,000 test takers from 158 different countries, speaking 126 different native languages. This data was collected and used to train the automated scoring engines for both the written and spoken PTE Academic items.

To do this, multiple trained human markers assess each answer. Those results are used as the training material for machine learning algorithms, similar to those used by systems like Google Search or Apple’s Siri. The model makes initial guesses as to the scores each response should get, then consults the actual scores to see well how it did, adjusts itself in a few directions, then goes through the training set over and over again, adjusting and improving until it arrives at a maximally correct solution – a solution that ideally gets very close to predicting the set of human ratings.

Once trained up and performing at a high level, this model is used as a marking algorithm, able to score new responses just like human markers would. Correlations between scores given by this system and trained human markers are quite high. The standard error of measurement between ɫèAV’s system and a human rater is less than that between one human rater and another – in other words, the machine scores are more accurate than those given by a pair of human raters, because much of the bias and unreliability has been squeezed out of them. In general, you can think of a machine scoring system as one that takes the best stuff out of human ratings, then acts like an idealized human marker.

ɫèAV conducts scoring validation studies to ensure that the machine scores are consistently comparable to ratings given by skilled human raters. Here, a new set of test-taker responses (never seen by the machine) are scored by both human raters and by the automated scoring system. Research has demonstrated that the automated scoring technology underlying PTE Academic produces scores comparable to those obtained from careful human experts. This means that the automated system “acts” like a human rater when assessing test takers’ language skills, but does so with a machine's precision, consistency and objectivity.

Scoring speaking responses with ɫèAV’s Ordinate technology

The spoken portion of PTE Academic is automatically scored using ɫèAV’s Ordinate technology. Ordinate technology results from years of research in speech recognition, statistical modeling, linguistics and testing theory. The technology uses a proprietary speech processing system that is specifically designed to analyze and automatically score speech from fluent and second-language English speakers. The Ordinate scoring system collects hundreds of pieces of information from the test takers’ spoken responses in addition to just the words, such as pace, timing and rhythm, as well as the power of their voice, emphasis, intonation and accuracy of pronunciation. It is trained to recognize even somewhat mispronounced words, and quickly evaluates the content, relevance and coherence of the response. In particular, the meaning of the spoken response is evaluated, making it possible for these models to assess whether or not what was said deserves a high score.

Scoring writing responses with Intelligent Essay Assessor™ (IEA)

The written portion of PTE Academic is scored using the Intelligent Essay Assessor™ (IEA), an automated scoring tool powered by ɫèAV’s state-of-the-art Knowledge Analysis Technologies™ (KAT) engine. Based on more than 20 years of research and development, the KAT engine automatically evaluates the meaning of text, such as an essay written by a student in response to a particular prompt. The KAT engine evaluates writing as accurately as skilled human raters using a proprietary application of the mathematical approach known as Latent Semantic Analysis (LSA). LSA evaluates the meaning of language by analyzing large bodies of relevant text and their meanings. Therefore, using LSA, the KAT engine can understand the meaning of text much like a human.

What aspects of English does PTE Academic assess?

Written scoring

Spoken scoring

  • Word choice
  • Grammar and mechanics
  • Progression of ideas
  • Organization
  • Style, tone
  • Paragraph structure
  • Development, coherence
  • Point of view
  • Task completion
  • Sentence mastery
  • Content
  • Vocabulary
  • Accuracy
  • Pronunciation
  • Intonation
  • Fluency
  • Expressiveness
  • Pragmatics

More blogs from ɫèAV

  • Students sat at a desk looking at a textbook together, with a teacher pointing to it

    Real-world English: How GSE Job Profiles bridge learning and work

    By Sara Davila
    Reading time: 5 minutes

    Did you know that improving your English proficiency can increase earning potential by up to 50% and safeguard your career against AI? Recent research by ɫèAV highlights that English is not just a skill but a career-defining advantage in today’s globalized workplace. For millions of adult learners, the journey from the classroom to the workplace requires more than general conversational abilities—it’s about gaining targeted, job-ready skills as quickly as possible. For English language educators, understanding what “jDz-𲹻” English is and how to identify “jDz-𲹻” skills can provide a significant advantage in ensuring learners are prepared to communicate effectively and collaborate with their future coworkers.

    That’s where the Global Scale of English (GSE) Job Profiles comes in. For educators and program developers, it offers a bridge between real-world job skills and the English learners need to perform them. Whether you’re creating programs for nursing assistants, hospitality workers, or IT professionals, this tool ensures that learners build the precise English skills they need to thrive in their roles.

    Let’s walk through how to create a GSE Job Profile and explore its practical use for building programs that align with today’s professional realities.

  • A older techer sat with students, in particular looking at a laptop with one of the students

    Teacher technology adoption: What’s age got to do with it?

    By Jennifer Williams
    Reading time: 4 minutes

    Educators, by design, are innovators. Living and working in a constant state of beta, teachers bring to the classroom a natural desire to explore and better understand new practices and methodologies to support students in their search for knowledge.

    Guided by an internal drive to make the world a better place and powered by the rewards of working daily with the youth that can create that positive change, teachers collectively are part of a profession different from any other. Within one classroom on any single day, a teacher can become a scientist, a historian, an inventor, or a scholar. Over time, whether in a one-room schoolhouse a century ago or in a technology enhanced classroom of the digital age, these defining characteristics of “teacher” have seemed to stay constant.

    Bridging the generational and digital divide

    Today, as citizens of our digital and global world, we are in an extraordinary time where divides are closing and collaboration becomes standard – where differences are truly celebrated and associations are surpassing separations in society.As educators, this change reflects what we have consistently asserted in our classrooms.

    Yet, terms like 'generational gap” and “digital divide” suggest a significant separation between us as teachers and our students. In truth, teachers and students are increasingly using asharedlanguage of innovation and exploration. Rather than sticking to notions of a technology-based generational divide, many are transcending the labels of “digital immigrants” and “digital natives”.

    This is further reinforced by current research demonstrating that technology adoption in education has less to do with age and instead is based more on exposure and experience (Bennett & Maton, 2010; Bullen, Morgan, & Qayyum, 2011; Guo, Dobson, & Petrina, 2008; Helsper & Eynon, 2010). As opposed to a focus on divides, this new evidence offers supportive pathways of possibilities to bring together older and younger generations in the process of learning in our world through exploration and wonder.

    Embracing skill development

    Moving attention away from a fixed age factor to emphasizing a continuum of proficiency based on exposure and experience enables teachers of all ages to identify as part of this digital age of education. Though digital technologies are fixtures in the daily lives of many (if not most) of our students, the ways students use technologies are not always consistent.

    Students of today – like students of previous generations – continue to need the guidance of teachers in finding ways to take and apply their understanding of technology to effectively convey perspectives with the world, influence the opinions of others, and contextualize and synthesize information in meaningful ways.

    Timeless lessons from teachers

    Teachers as innovators – young and old, tech-savvy and tech-novice – have extraordinary gifts to bring to their students, who are also navigating through digital environments. Though there are countless lessons teachers bring to students, here are several timeless ones that transcend any generation of teaching. Teachers guide students to use advanced technologies so that they can be...

    Critical consumers

    Today, students in our classrooms are bombarded with information at nearly every moment in time. With basic searches for information often leading students in multiple directions, research at every level has now become a highly complex and sophisticated process. The result is that we now see learning practices shifting from finding answers to discovering more questions.

    Teachers can guide students to evaluate and consume information critically. By promoting transparency and sharing, they help learners consider context and sources, reason with evidence and build knowledge.

    Creators of rich content

    As classrooms advance as blended learning environments, seamless integration of technologies and innovative teaching practices offers students opportunities to transform from passive learners to active content creators. With countless digital tools available for content creation, students can seek guidance from teachers on matching the best tool to a particular communicative purpose.

    Teachers who are skillful in creating synthesized messages can show students ways to comprehend and compose with combinations of digital technologies. Less in the role of directing lessons, the teacher becomes a catalyst for learning, inspiring students to work creatively and collaboratively to solve problems and respond to instructional lessons as producers of information.

    Centered on relationships

    A defining quality of a master teacher is the ability to create a positive learning community built on empathy and understanding. The focus on relationships in these classrooms models the continued importance of valuing alliances and friendships.

    Soft skills, such as joy, honesty, trust and respect, become even more essential as our students continue to interact in a highly networked and diverse world. Balance, too, becomes a critical area for discussion as students look to divide attention, time and social-emotional reserves between onscreen and in-person experiences.

    Within responsive classrooms and digital learning spaces that keep emphasis on the formation of relationships, teachers can offer opportunities for students to develop through collaboration, teamwork, solidarity and conflict resolution.

    Resilient in endeavors

    Our students are looking out onto a world where anything is possible. Digital age technologies are empowering students to believe: “if you can dream it, you can do it”. The impossible is within reach, and a mindset of grit and determination moves sparked ideas to a place of accomplishment and excellence.

    Through the creation of learner-driven classrooms centered on the student, teachers can help guide students to narrow their interests and ultimately find their passions. With relevance and authentic purpose, teachers can connect students in the process of learning through design thinking and trial and error – shifting the outlook from fear of failure to resilience and 'stick-to-itiveness' to advance learning to a place of discovery and invention.

    And it is this mindset – a position of inquiry and exploration – held by students and teachers, young and old, that can show that great will come from risk. We all just need to be ready to try.

  • A man reading a book sat in a window

    Ways to help you work out the meaning of new words

    By
    Reading time: 4 minutes

    Learning a new language is an exciting journey, but it can also be challenging, especially when you encounter unfamiliar words. Don't let new vocabulary intimidate you. With the right strategies, you can work out the meaning of new words and enhance your language skills. Here are some effective ways to help you work out the meaning of new words so you can use these new words confidently and tackle any unfamiliar words with confidence.

    Before diving into practical tips, it's helpful to understand a bit about the origins of the English language. English is a Germanic language, but it has been heavily influenced by Latin, especially through the Norman Conquest in 1066 and the Renaissance period. Many English words have Latin roots, prefixes and suffixes. This historical context can be a valuable tool in decoding unfamiliar vocabulary.