Explaining computerized English testing in plain English

ɫèAV Languages
a pair of hands typing at a laptop

Research has shown that automated scoring can give more reliable and objective results than human examiners when evaluating a person’s mastery of English. This is because an automated scoring system is impartial, unlike humans, who can be influenced by irrelevant factors such as a test taker’s appearance or body language. Additionally, automated scoring treats regional accents equally, unlike human examiners who may favor accents they are more familiar with. Automated scoring also allows individual features of a spoken or written test question response to be analyzed independent of one another, so that a weakness in one area of language does not affect the scoring of other areas.

was created in response to the demand for a more accurate, objective, secure and relevant test of English. Our automated scoring system is a central feature of the test, and vital to ensuring the delivery of accurate, objective and relevant results – no matter who the test-taker is or where the test is taken.

Development and validation of the scoring system to ensure accuracy

PTE Academic’s automated scoring system was developed after extensive research and field testing. A prototype test was developed and administered to a sample of more than 10,000 test takers from 158 different countries, speaking 126 different native languages. This data was collected and used to train the automated scoring engines for both the written and spoken PTE Academic items.

To do this, multiple trained human markers assess each answer. Those results are used as the training material for machine learning algorithms, similar to those used by systems like Google Search or Apple’s Siri. The model makes initial guesses as to the scores each response should get, then consults the actual scores to see well how it did, adjusts itself in a few directions, then goes through the training set over and over again, adjusting and improving until it arrives at a maximally correct solution – a solution that ideally gets very close to predicting the set of human ratings.

Once trained up and performing at a high level, this model is used as a marking algorithm, able to score new responses just like human markers would. Correlations between scores given by this system and trained human markers are quite high. The standard error of measurement between ɫèAV’s system and a human rater is less than that between one human rater and another – in other words, the machine scores are more accurate than those given by a pair of human raters, because much of the bias and unreliability has been squeezed out of them. In general, you can think of a machine scoring system as one that takes the best stuff out of human ratings, then acts like an idealized human marker.

ɫèAV conducts scoring validation studies to ensure that the machine scores are consistently comparable to ratings given by skilled human raters. Here, a new set of test-taker responses (never seen by the machine) are scored by both human raters and by the automated scoring system. Research has demonstrated that the automated scoring technology underlying PTE Academic produces scores comparable to those obtained from careful human experts. This means that the automated system “acts” like a human rater when assessing test takers’ language skills, but does so with a machine's precision, consistency and objectivity.

Scoring speaking responses with ɫèAV’s Ordinate technology

The spoken portion of PTE Academic is automatically scored using ɫèAV’s Ordinate technology. Ordinate technology results from years of research in speech recognition, statistical modeling, linguistics and testing theory. The technology uses a proprietary speech processing system that is specifically designed to analyze and automatically score speech from fluent and second-language English speakers. The Ordinate scoring system collects hundreds of pieces of information from the test takers’ spoken responses in addition to just the words, such as pace, timing and rhythm, as well as the power of their voice, emphasis, intonation and accuracy of pronunciation. It is trained to recognize even somewhat mispronounced words, and quickly evaluates the content, relevance and coherence of the response. In particular, the meaning of the spoken response is evaluated, making it possible for these models to assess whether or not what was said deserves a high score.

Scoring writing responses with Intelligent Essay Assessor™ (IEA)

The written portion of PTE Academic is scored using the Intelligent Essay Assessor™ (IEA), an automated scoring tool powered by ɫèAV’s state-of-the-art Knowledge Analysis Technologies™ (KAT) engine. Based on more than 20 years of research and development, the KAT engine automatically evaluates the meaning of text, such as an essay written by a student in response to a particular prompt. The KAT engine evaluates writing as accurately as skilled human raters using a proprietary application of the mathematical approach known as Latent Semantic Analysis (LSA). LSA evaluates the meaning of language by analyzing large bodies of relevant text and their meanings. Therefore, using LSA, the KAT engine can understand the meaning of text much like a human.

What aspects of English does PTE Academic assess?

Written scoring

Spoken scoring

  • Word choice
  • Grammar and mechanics
  • Progression of ideas
  • Organization
  • Style, tone
  • Paragraph structure
  • Development, coherence
  • Point of view
  • Task completion
  • Sentence mastery
  • Content
  • Vocabulary
  • Accuracy
  • Pronunciation
  • Intonation
  • Fluency
  • Expressiveness
  • Pragmatics

More blogs from ɫèAV

  • A woman holding a book to her face, reading in a bookstore with shelves of books behind her

    Fantasy, the English language and Tolkien

    By
    Reading time: 6 minutes

    A large number of well-known writers have often created or coined words that are used in everyday English. When you think of authors, prominent figures like Shakespeare may come to mind. He enriched the English language with words like "amazement," "bedazzled," and "fashionable." Charles Dickens introduced "boredom," showcasing his talent for capturing profound human emotions and societal issues in a single word. Lewis Carroll added whimsical words to our lexicon, including "chortle," a delightful mix of 'chuckle' and 'snort.'

    But Tolkien is another one of those authors who has added to the English language's colorful dictionary. Tolkien did not just create worlds; he also enriched our language, adding a lexicon that elicits the smell of mead in crowded halls and the sight of smoky mountains veiled in mystery. Language enthusiasts and fantasy fans alike join us on this philological adventure as we uncover the words that J.R.R. Tolkien, the mastermind behind Middle-earth, either coined or brought into the limelight.

  • A teacher stood by two young students looking at a computer monitor

    The potential of AI in English language teaching

    By
    Reading time: 6 minutes

    The integration of Artificial Intelligence systems (AI) into English language teaching represents a significant shift in educational methodologies. This emerging technology offers English teachers a myriad of opportunities to enhance their teaching strategies, making the learning process more engaging, personalized and effective. In this blog post, we will explore practical tips on how AI can aid English language teaching and empower both educators and learners.

    One noteworthy reference highlighting AI's impact on education is the report by the Organisation for Economic Co-operation and Development (OECD), ''. This comprehensive study outlines how AI technologies, assists administrative and assessment aspects of teaching but also revolutionizes the way students learn. AI tools are paving the way for a more adaptable and learner-centric approach in English language teaching by offering bespoke, adaptive learning pathways and instant feedback.

    How is AI currently being used in schools?

    Currently, schools are just beginning to harness the power of AI to enhance English language learning in several innovative ways. One notable application of educational technology is the use of intelligent tutoring systems, which provide students with personalized feedback and learning paths tailored to their individual needs and proficiency levels.

    Additionally, AI-driven language learning apps and platforms have become increasingly popular, offering interactive and immersive learning experiences through natural language processing and machine learning technologies.

    These platforms can simulate conversation practice, offer pronunciation correction, and even adapt the content in real-time to challenge students appropriately.

    Furthermore, AI is also being used for administrative tasks, such as grading and assessing students' work, allowing teachers more time to focus on curriculum development and one-on-one student interaction. This integration of AI into English language teaching is enhancing the efficiency of learning processes and actively contributing to a more engaging and dynamic educational environment.

    How AI can enhance English language teaching

    Personalization at scale

    AI systems can analyze individual student learning styles and preferences, allowing for personalized lesson plans that cater to the unique needs of every student. By customizing content, pacing and learning activities, AI ensures that students remain engaged and receive targeted support, significantly improving learning outcomes.

    Interactive learning experiences

    AI-powered applications, educational games and tools can create immersive and interactive language learning experiences. From chatbots that simulate conversation, to platforms that offer real-time feedback on pronunciation, these tools can help students to practice speaking and listening skills in a controlled and safe environment outside the traditional classroom setting.

    Autonomous learning support

    With the assistance of AI, students can self-study more effectively. AI tutoring systems can provide instant feedback on written work, ensuring learners can progress even when a teacher isn't immediately available to teach. These systems offer consistent, unbiased support, which is invaluable for building students' confidence.

    Enhanced assessment capabilities

    Assessment is a crucial part of the learning process. AI can take on the laborious task of grading and provide detailed insights into a student's performance. Teachers can then use this data to identify areas where students struggle and tailor future instruction to address these gaps.

    Expanding the creative horizons

    AI's applications extend into creative writing, offering students prompts and suggestions to overcome writer's block and develop storylines. This enhances creativity and motivation in students by providing them with a springboard for their writing skills.

    Improving your own material

    It is beneficial to utilize tools to refine your writing by adjusting tone, style, and paraphrasing. These tools are particularly useful for crafting model answers and providing feedback to students. You can also make that material more visually appealing with generated images.

    Utilizing ChatGPT in language teaching

    ChatGPT, an AI language model, can aid and save time on the way language lessons are conducted by creating a highly interactive and responsive environment for students. Teachers can harness this technology to simulate real-life conversations, enabling students to practice their language skills in a dynamic setting. Students can also be encouraged to use it to start first drafts and use their critical thinking.

    By inputting specific scenarios or topics, ChatGPT can generate dialogues that challenge and teach students how to use new vocabulary and grammar structures in context, bridging the gap between theoretical learning and practical application. Furthermore, its capacity to provide immediate feedback allows learners to correct their mistakes in real time, fostering a learning atmosphere that is both efficient and encouraging.

    The versatility of these kinds of AI chatbots means they can be tailored to suit learners at different proficiency levels, making them an invaluable tool for language teachers aiming to enhance engagement and facilitate deeper learning.

    Tips for teachers integrating AI in English lessons

    1. Start with a clear goal: define what you aim to achieve by incorporating AI into your lessons.
    2. Combine traditional and AI methods: use AI as a complement, not a substitute, for human interaction.
    3. Prioritize privacy and ethics: ensure any AI tools used are compliant with privacy laws and ethical standards.
    4. Stay updated: AI is a fast-evolving field. Attend professional development webinars and workshops to stay current.
    5. Foster a growth mindset: encourage students to view AI as a tool to aid their own effort and perseverance.
    6. Demystify technology: explain how AI works, alleviating any concerns or misapprehensions about its use.
    7. Experiment and iterate: not every AI application will suit your classroom – be prepared to try different tools and approaches.

    What about cheating and plagiarism with AI?

    The issue of cheating and plagiarism is a significant concern in our higher education institutions and is becoming more pronounced with the advent of AI technologies. However, AI itself can be a formidable ally in combating these challenges. AI-powered tools can analyze student submissions to detect plagiarism effectively, providing educators with robust mechanisms to ensure the integrity of academic work. Additionally, AI systems can be programmed to recognize students' unique writing styles, making it easier to identify discrepancies that suggest dishonesty.

    It's important for educators to discuss these topics openly with students, emphasizing the value of originality and the serious consequences of academic misconduct. By leveraging AI not just for educational enhancement but also as a means of maintaining academic standards, educators can foster a culture of honesty and integrity within the classroom.

    Learn more to boost your knowledge of AI

    Billie Jago’s Learn with Me Series

    Explore Billies' series of free videos and lesson plans for 101 on using AI in the classroom. These are for educators who want to get to grips with the basics of AI fast, with practical tips that can be applied immediately in the classroom.

    They will be released throughout August and September 2024.

    and check .

    Building confidence inAI skills for English language teaching webinar

    To fully unlock the potential of AI in the world of English language teaching, continuous learning is a must. We're excited to introduce an upcoming webinar series 'Building confidence inAI skills for English language teaching' that will guide English teachers through the nuances of AI.

    From the 23rd to 27th of September 2024, join us for a series of engaging webinars designed for English teachers.

    Benefit from insights shared by experts. Plus, earn a Credly by ɫèAV badge for your participation in all of the sessions.

    Join us as we explore the frontiers of AI in education and discuss how to prepare for the evolving educational landscape. It's time to equip yourself with the knowledge and tools required to thrive in an AI-influenced educational environment.

  • Two women in a pottery room talking over a laptop

    Cutting through corporate English: Clearer alternatives to business jargon

    By
    Reading time: 4 minutes

    The corporate world can often feel like an echo chamber of confusing phrases. Terms like "blue-sky thinking" and "tiger teams" might dominate conversation in boardrooms and email chains, but what do they actually mean in practice and are they the best way to communicate?

    This blog post simplifies workplace English by offering straightforward business English alternatives to common corporate jargon. These replacements will resonate with fluent English language speakers and ease communication with non-fluent English speakers who are vital contributors to the workplace. Plus, a clearer form of communication can help foster inclusivity, improve understanding and ultimately enhance productivity.

    What is corporate jargon?

    Corporate jargon refers to the very specialized vocabulary and language used within the business world, often characterized by buzzwords, acronyms, jargon and phrases that are usually unknown to outsiders.

    Why should we avoid professional jargon?

    Jargon often acts as a barrier to clear communication, creating an environment where messages are misunderstood or misinterpreted. This misunderstanding can lead to inefficiencies, errors and frustrations among team members, particularly those who may not be fluent English speakers or are new to the corporate environment.

    Furthermore, overusing complex language can unintentionally exclude or push away parts of the workforce, undermining efforts to build an inclusive and collaborative team culture. In essence, simplifying language and avoiding unnecessary jargon can make available information more accessible to a wider audience and ensure that all team members have the necessary information to contribute effectively to their roles.

    For example, let's look at "think outside the box." It means to think in new ways, but it's said so much that it doesn't really grab attention anymore. A better way to say this might be "think creatively" or "come up with new ideas." Also, the word "synergize" really just means to work well together for better results. Saying "work together effectively" is much clearer and easier for everyone to understand, no matter if they're used to business terms or not. Using simple words helps everyone get the idea faster, making the workplace more welcoming and efficient.

    The difference between English jargon and slang

    Jargon functions as a shorthand among professionals, representing complex concepts among those in the know. Slang is less formal than jargon, often coming from cultural or societal trends and used more in casual conversations.

    An example of jargon vs slang

    An example of jargon and slang could be comparing "low-hanging fruit" to "no-brainer." "Low-hanging fruit" is a piece of jargon that refers to tasks, projects or targets that are easily achievable and do not require much effort.

    On the other hand, "no-brainer" is slang that implies something is an obvious or easy decision, often used in more informal settings. While both terms convey the idea of simplicity and ease, "low-hanging fruit" is more likely to be used in strategic meetings or project planning sessions, whereas "no-brainer" might be used in informal discussions or brainstorming sessions.

    Examples of jargon and their plain English alternatives

    Blue-sky thinking

    Avoid saying, "We need some blue-sky thinking." Instead, use,"Approach this problem with unlimited creativity."

    Helicopter view

    Avoid saying,"Give me a helicopter view of the situation." Instead, use,"Provide a broad overview of the situation."

    Get all your ducks in a row

    Avoid saying, "Before the busy season, get all your ducks in a row." Instead, use,"Ensure you're well-organized before the busy period begins."

    Thinking outside the box

    Avoid saying, "This client expects us to think outside the box." Instead, use,"The client expects unique and unconventional ideas from us."

    IGUs (Income Generating Units)

    Avoid saying,"Our goal is to increase our IGUs this quarter." Instead, use,"Our goal is to increase our student enrolments this quarter."

    Run it up the flagpole

    Avoid saying,"That's a great idea. Run it up the flagpole." Instead, use,"That's a great idea. Share it with the team for their input."

    Swim lanes

    Avoid saying, "To understand your tasks, refer to the swim lanes." Instead, use,"Refer to the job responsibility diagram to understand your tasks."

    Bleeding edge

    Avoid saying, "This new software is bleeding edge." Instead, use,"This new software represents the latest in research and technological innovation."

    Tiger team

    Avoid saying,"We'll need a tiger team to tackle this project." Instead, use,"We'll assemble a specialized task force for this project."

    While some professionals may use jargon to appear more knowledgeable, the truth is that simplicity is key. Clear and straightforward language not only levels the playing field for everyone but also cuts through possible misunderstandings.

    The case for plain business English

    In today's global business environment, where interactions occur across diverse linguistic backgrounds and languages, it's paramount to ensure that our words are as clear and simple as possible. By favoring plain English vocabulary over opaque jargon, we promote an inclusive and efficient workplace where every member, fluent or not, understands and contributes to their fullest potential.

    Continue to seek clarity in your communications. Remember that the true mark of expertise is not using complex language but making complex concepts understandable for everyone. Keep challenging yourself to simplify your language without diluting the meaning of your message. Remember, whether you're a seasoned professional or a new recruit, clear communication will always be your strongest asset.

    If you or your team are looking to improve their business English skills, check out our business English courses and blog posts for up-to-date tips.