Wednesday, 18 November 2026

Why We Tested TeacherMatic When We Already Had Copilot

Rae Karimjee, Senior Educational Technologist at City St George's, writes about her university's participation in the BLE's Jisc-supported pilot of the platform,TeacherMatic. The post was originally published here.

As generative AI becomes increasingly embedded in higher education, many universities are wrestling with the same question: do specialist AI tools offer something genuinely different, or can the AI tools we already have meet most of our needs?

At City St George’s, staff already have access to Microsoft Copilot alongside a growing range of AI-enabled tools. So when we joined the TeacherMatic pilot through the Bloomsbury Learning Exchange (BLE) and Jisc, our aim wasn’t to find our first AI solution. Instead, we wanted to understand whether a specialist educational AI platform could offer additional value and whether it would support the varied assessment, feedback and curriculum design needs across our Schools.

TeacherMatic is designed specifically for educators, with tools intended to support activities such as assessment design, feedback generation, rubric creation, lesson planning and quiz development. The pilot involved five London institutions and gave us the opportunity to evaluate the platform in authentic teaching and assessment contexts rather than relying on vendor claims alone. 

What Were We Hoping to Learn?

During an initial check-in with pilot participants, four key expectations emerged. Staff wanted to know whether TeacherMatic could:

  • Make feedback more consistent.
  • Speed up marking. 
  • Support the creation of assignment briefs and rubrics. 
  • Reduce workload more broadly. 

These are challenges familiar to almost anyone involved in teaching and assessment. The evaluation therefore focused not simply on whether TeacherMatic could generate content, but whether it could improve the overall assessment and feedback workflow. 

Early Enthusiasm, Limited Engagement

Interest in the pilot was encouraging, but one of the findings that emerged throughout the project was the difference between interest in AI and active use of an AI tool. More than half of participants had not yet started using TeacherMatic at the point of the first formal review, and only a small number had undertaken significant testing. 

This proved to be an important lesson. A successful educational technology needs not only to demonstrate capability but also to fit naturally into busy academic workflows. If staff cannot easily see where a tool fits into their day-to-day practice, adoption is likely to remain limited regardless of its potential. 

What Staff Found

For colleagues who tested the platform, the picture was mixed. Several participants appreciated the ability to generate structured outputs quickly. The multiple-choice question (MCQ) generator, for example, showed promise as a starting point for question development, helping academics create initial drafts that could then be refined for their discipline and level of study. 

The Advanced Feedback tool also attracted interest. Staff found it could produce detailed feedback aligned to rubric criteria and generate draft comments significantly faster than they could be written manually.  However, several recurring concerns emerged:

  • Significant prompting and editing were often required. 
  • Feedback could feel generic or recognisably AI-generated. 
  • Rubrics were not always applied consistently. 
  • Hallucinated references and inaccuracies occasionally appeared. 
  • Outputs were not always appropriate for higher-level academic work. 

For some participants, the challenge wasn’t that the platform couldn’t produce outputs. It was that time saved during content generation was often replaced by time spent reviewing, checking and editing the results. Academic judgement remained essential throughout the process. 

The Workflow Challenge

One of the strongest themes to emerge from the pilot was the importance of workflow. Academic colleagues working with larger cohorts highlighted practical requirements such as:

  • Bulk upload of coursework from Moodle. 
  • Efficient generation of feedback at scale. 
  • Retention of anonymised student identifiers. 
  • Seamless mapping of feedback back to students. 

Questions arose about whether TeacherMatic could reliably support these processes without adding manual steps. For colleagues marking hundreds of submissions, even small workflow inefficiencies could offset any advantages offered by AI-generated feedback. Another participant found they were unable to use the platform effectively for coursework submitted in Excel format. 

These findings reinforced an important point: the effectiveness of an AI tool is about far more than the quality of its outputs. Integration into existing academic practice matters just as much.

A Closer Look at Assessment Feedback

One of the most detailed evaluations came from Luis da Vinha, Senior Lecturer in International Politics and a Digital Champion, who compared TeacherMatic’s Advanced Feedback tool with human marking across 41 undergraduate assignments. The results highlighted both strengths and limitations. On the positive side, TeacherMatic generated structured, rubric-aligned feedback considerably faster than human markers. This demonstrated clear efficiency benefits and suggested potential value in supporting feedback preparation and formative activities. However, the evaluation also found that:

  • TeacherMatic awarded marks that averaged 4.17 percentage points higher than human markers. 
  • Grades tended to be compressed towards the middle range. 
  • Feedback sometimes lacked module-specific context. 
  • The tool could not replicate the disciplinary insight and contextual understanding provided by experienced academics. 

The conclusion was clear: while TeacherMatic could support feedback preparation, it should not be used for autonomous summative marking. Human oversight remains critical. 

Comparing TeacherMatic with Existing AI Tools

Perhaps the most important question throughout the pilot was not whether TeacherMatic worked, but whether it offered enough additional value over AI tools already available to staff.

Several participants noted that experienced users of Microsoft Copilot or ChatGPT could often achieve similar outcomes through effective prompting. In this context, TeacherMatic’s main advantage appeared to be the educational workflows and templates wrapped around the AI, rather than consistently superior outputs. 

One participant who explicitly compared TeacherMatic with existing AI tools concluded that, aside from some potential benefits around rubric-based feedback generation at scale, the platform offered little improvement over the tools staff already had available. 

Our Decision

Having reviewed the evidence gathered throughout the pilot, City St George’s decided not to continue with TeacherMatic at this time. 

This decision was not based on any single evaluation or participant experience. The platform demonstrated genuine strengths, including rapid content generation, structured outputs and support for some drafting and formative activities. 

However, the anticipated benefits around workload reduction, assessment and feedback quality were not demonstrated consistently enough to justify continued adoption. Across the pilot, concerns remained around academic judgement, workflow compatibility, disciplinary context, marking reliability, engagement levels and overall value when compared with existing AI provision. 

What We Learned

While we won’t be taking TeacherMatic forward, the pilot has been extremely valuable. It highlighted five areas we will continue to consider when evaluating future AI tools:

  • Pedagogical value. 
  • Support for academic judgement. 
  • Workflow integration. 
  • Whole-workflow efficiency, not just generation speed. 
  • Additional value beyond existing AI provision. 

Perhaps the biggest lesson is that generating content quickly is not the same as improving educational practice. As AI technologies continue to evolve, we need to evaluate them not only on what they can produce, but on whether they genuinely enhance teaching, assessment and the student experience. 

The TeacherMatic pilot helped us answer one question about a specialist AI platform, but it also raised a broader one for our institution:

Are we looking for AI tools that can generate content faster, or tools that can genuinely improve the way we teach, assess and support learning?

The answer to that question will likely shape our future AI decisions far more than any individual product evaluation.

Further Reading

The full evaluation report provides a detailed account of the pilot methodology, participant feedback, assessment findings and recommendations.  Readers interested in exploring the evidence in more depth are encouraged to read the full report alongside the wider findings from the Bloomsbury Learning Exchange collaboration. ​Jisc TeacherMatic BLE Pilot Project report

Tuesday, 17 November 2026

Demystifying the PhD process

The BLE Team has just updated our popular course, “Is a PhD Right for Me?”. If you’re not already familiar with it, Tom Graham shares what it’s all about in this post:

Many undergraduate and postgraduate students can find the whole PhD application process intimidating or even baffling. In-person events where potential applicants can ask questions and discuss their issues face-to-face with current doctoral researchers and supervisors can, of course, be of enormous benefit.

However, for some potential applicants these kinds of events can pose challenges of their own. There may be concerns about travel costs, time taken off from paid employment, and childcare arrangements, as well as issues involving physical disabilities and neurodivergence. Even factors such a low self-confidence and perceptions about ethnicity and social class can all play in to compelling an individual to tacitly self-censor their PhD application before they have even started.

Image of a PhD student working on a laptop
Prompted by our members to help address this issue, the Bloomsbury Learning Exchange surveyed over 200 PhD students about their application journey. Most felt that they had started their PhD underinformed about what it would entail, with many regretting their lack of preparation for the intensity of doctoral study.

Guided by these survey results and garnering the input of a range of academics, support service staff and students, we created “Is a PhD Right for Me?”, a Massive Open Online Course (MOOC) on the FutureLearn platform. Our MOOC seeks to empower potential candidates to make informed decisions about whether pursuing a PhD would be the right option for them, if it would be right then how best to go about it and what challenges to expect throughout the whole experience.

The MOOC format avoids the issues that might deter some people from attending in-person events. Students can access the course from home and at whatever times best suit them, working at their own pace and focusing on whatever aspects they find most relevant.

Student studying at a laptop
We developed the content to not only clarify what a PhD is but to allay concerns which may cause potential candidates to pre-emptively talk themselves out of applying before taking even their first step:  personal circumstances concerning race, disability, and gender may compel an individual to tacitly self-censor their PhD application, but so too might issues about age, psychological wellbeing, and insecurities about intellectual fitness to engage in high level academic study. Our intent was proactively to make clear that none of these factors automatically debar a potential PhD applicant or prejudice their academic potential.

Participants who have compl
eted the whole course have told us how much it had empowered them to make a better-informed decision about whether or not to proceed with a PhD application:
“I am disabled and haven't been in paid employment for many years due to ill health. Now I feel more confident on the application process and the time management aspects in particular. This course has been really helpful - thank you!”

It has also changed how some potential applicants see the more personal aspects of PhD study: “This course overturned the stereotypes I had. I learned that there could be PhD students who are easy-going and enjoy life and work and who are not serious all the time.”

By demystifying the whole PhD process – from initial application to daily life as a researcher – we hope that our MOOC informs and emboldens a diversity of potential PhD candidates who otherwise may have written themselves off before they had even begun.

Friday, 30 October 2026

Digital Accessibility Starts at Procurement

Take from this original post authored by Xavier Briche and publised by ALT, the Association for Learning Technology.

The second presentation, delivered by Sarah Sherman, Director at Bloomsbury Learning Exchange explored the challenges of embedding digital accessibility into procurement. Using real supplier responses to accessibility questions, Sarah highlighted common issues, including accessibility being offered at extra cost, reliance on overlays rather than accessible design, poor understanding of legal obligations, and accessibility features being absent from product roadmaps.

The session highlighted how accessibility testing often occurs late in procurement, placing significant demands on internal specialists. Participants also discussed how suppliers frequently focus on student-facing interfaces while overlooking accessibility barriers in staff-facing workflows.

A central theme of the presentation was the Accessibility Passport, developed through the Make Things Accessible project. Designed as a practical framework for procurement, it provides shared tools and resources that institutions can adopt rather than developing their own approaches independently.

Resources include:

  • Standard contract wording
  • Procurement guidance
  • Tender evaluation criteria
  • Accessibility matrices
  • Eight key questions for suppliers

The aim is to create greater consistency across the sector, helping institutions set clear expectations and encouraging suppliers to improve accessibility standards.

Discussion focused on the benefits of collaboration between institutions and the need for stronger partnerships between accessibility specialists, learning technologists, procurement teams and senior leaders. Participants agreed that accessibility must be considered throughout the procurement lifecycle rather than treated as a final compliance check.

Friday, 23 October 2026

Why assessment in the AI age doesn't have to go the way of the dinosaurs

 Originally published on the QAA website on 13th October 2026

Professor Mark J.P. Kerrigan

Director of the SOAS Academy of Learning and Teaching (SALT)  and Professor of Educational Innovation and Futures, School of Oriental and African Studies, University of London

 

Dr Sian Lindsay

Academic Developer, University of Reading

 

Sarah Sherman

Director, The Bloomsbury Learning Exchange


There’s that well-known phrase that goes something like - ‘just because we can do something, doesn’t mean we should’. Mark wanted to know its origins, so he turned to Google for the answer and was rapidly bombarded with AI responses all trying to be helpful. After scrolling past all the AI noise, he finally found something real to reference - a quote from the 1993 movie Jurassic Park directed by Steven Spielberg. Actor Jeff Goldblum plays a sceptical scientist, who famously quips “Yeah but your scientists were so preoccupied with whether they could, they didn’t stop to think if they should”. Whilst we’re not trying to suggest that the explosion of AI into our lives poses exactly the same existential threat as bringing dinosaurs back from extinction(!), we have had to change and evolve the way we do things significantly and rapidly, with very little time to prepare or reflect. AI is everywhere, it can be wonderfully helpful, providing access to knowledge, but it can also really confuse things.


AI has challenged all industries and sectors, and its impact on education – in particular assessment – has been a growing concern. AI has intensified long-standing challenges and has introduced some new ones. We know that colleagues involved in the design, delivery and marking of assessments seek guidance and practical support in this space. Many (including us) would love to know how we can start it all again - design assessments that are inclusive, uphold academic standards, develop attributes that employers want and assure learning in the AI era. Unfortunately, there is no magic bullet that can address all these aspects perfectly at once, and compromises may be necessary (Corbin et al., 2025). To address this concern, SOAS is leading a QAA Collaborative Enhancement Project with six other HE institutions and the Bloomsbury Learning Exchange (BLE) to pilot, evaluate, and refine a practical Assessment Toolkit that supports educators in designing secure, inclusive, AI-aware, and future-focused assessment (Figure 1). 

SOASpic1

Figure 1: Overview of the project

The project recognises that generative AI represents a “wicked problem” for HE: complex, evolving and never definitively solvable (Corbin et al., 2025). As TEQSA (2025) argues, assessment reform must move beyond detection toward programme-level assurance of learning through coherent, evidence- informed assessment systems. The Assessment Toolkit operationalises this by providing structured workflows and information that help programme teams secure learning strategically, rather than reactively. The project advances existing practice by bridging the gap between quality, policy and day-to-day curriculum work. Our final project outputs will include a refined toolkit, practical resources and case studies. 


In this blog post, we thought it would be useful to share what the current version of the toolkit looks like. The original toolkit at SOAS University of London is a series of interlinked PowerPoint slide decks which include (1) a Home Deck, (2) a Companion Deck and (3) six slide decks spanning different top-level assessment approaches underpinned by specific examples (Table 1). The Home Deck comprises information and general instructions, and the Companion Deck provides information on feedback approaches based on the size of the student cohort. 


SOASpic3

Table 1: Assessment types included in the toolkit, grouped into six broad approaches


For each assessment type, the toolkit brings together analyses and practical resources to support its selection, design and use (Figure 2). These provide a conversation starter for educators and programme teams to review existing assessments, explore alternatives and discuss what will work in their context. The analyses help users consider alignment with learning outcomes and examine strengths and limitations from student, educator and generative AI perspectives. Example assignment briefs, rubrics, formative checkpoints and feedback ideas offer starting points for adaptation. They also prompt discussion about expectations for GenAI use and opportunities to develop AI literacy and relevant graduate attributes. In doing so, the toolkit responds to growing calls to rebalance assessment, giving greater attention to the processes of learning alongside the final product. It encourages educators to consider how students’ developing understanding, judgements, responses to feedback and reflections can become more visible and valued within assessment.


soaspic2

Figure 2: Analysis for each assessment modality


So, what have we been doing since the project started earlier this year? We have completed the project setup, created and shared a version of the toolkit with all our partners, and are recruiting programme and module teams to take part in the pilot. We have applied for and received ethical approval to conduct questionnaires and focus groups with participants, which will help us understand their experiences of the toolkit and inform future refinements before we share it at the end of the project. We have also presented our work at the SOAS Learning & Teaching Conference and the recent QAA online roadshow. We’ve been busy! 


Over the next few months, we will continue recruiting new participants and begin piloting and evaluating the toolkit - we’ll keep you updated on how we are getting along. 


References

Corbin, T., Bearman, M., Boud, D., & Dawson, P. (2025). The wicked problem of AI and assessment. Assessment & Evaluation in Higher Education, 1–17. https://doi.org/10.1080/02602938.2025.2553340 [Accessed 5 October 2026].

 

Tertiary Education Quality and Standards Agency (TEQSA), 2025. Enacting assessment reform in a time of artificial intelligence. Canberra: TEQSA. Available at: https://www.teqsa.gov.au/sites/default/files/2026-06/enacting-assessment-reform-in-a-time-of-artificial-intelligence-web.pdf [Accessed 5 October 2026].

Wednesday, 20 May 2026

Smart Glasses in Higher Education: Should we be worried or excited?

Authored by Elisabetta Lando, Miranda Melcher and Dom Pates from City St George's, University of London and originally posted here.

Summary

Smart glasses powered by AI and AR are gaining renewed interest in education, with potential for hands-free, guided, and immersive learning. City St George’s educational technologists ponder if the best use of this technology should be in targeted use cases only and even used Ms co-pilot to help produce this blog.