AI can give you a mark in seconds. But how much can you really trust that mark?

I am increasingly concerned about accountancy, tax and finance students using AI tools such as ChatGPT to mark their professional exam practice questions.

I understand the attraction. You complete a challenging past paper question, upload your answer, provide the marking scheme and ask AI to mark it. Within seconds, you receive detailed feedback and a percentage. It feels like you have received an independent assessment of your performance.

But here's the problem. AI can generate a mark. That doesn't mean it can mark your answer to the standard required by your professional examining body. And that distinction really matters.

Professional exam marking is about more than technical knowledge

Having worked as a lecturer and examiner in professional accountancy education, I know that marking involves much more than checking whether certain technical points appear in an answer.

Professional examinations assess your ability to interpret information, apply technical knowledge, exercise judgement and respond to the specific requirements of a question.

You can write several paragraphs of technically correct information and still earn relatively few marks because you haven't answered what was actually asked. Equally, you may approach a question differently from the suggested solution and still demonstrate the understanding required to earn marks.

Marking requires judgement about the quality, relevance and application of an answer.

An AI tool may produce a convincing assessment, but that doesn't mean it has consistently applied the examining body's marking conventions or standards.

The danger of trusting an AI-generated percentage

Consider a student preparing for an advanced taxation examination. They attempt a question involving a particular tax relief. Their answer explains the relevant legislation accurately but fails to apply several important conditions to the client's circumstances.

AI may recognise the correct technical content and award a reasonably strong mark. However, the examiner's marking scheme may place significant emphasis on applying that knowledge to the scenario. The student may therefore receive a much lower mark than anticipated.

The reverse can also happen. AI may underestimate a valid answer simply because it differs from the suggested solution. Neither outcome helps students accurately assess their readiness. And this brings me to another concern.

Are students looking for assessment or reassurance?

Professional exams come with considerable pressure.

When you've spent months studying, particularly if you've previously failed an exam, it's understandable that you want reassurance you're making progress. An AI-generated mark can provide that reassurance almost instantly. But reassurance is not the same as reliable evidence of exam readiness. A positive mark may encourage you to move on from a topic you haven't fully mastered. A disappointing mark may undermine your confidence unnecessarily.

Both can influence your study decisions. The issue isn't simply whether AI gives you the correct percentage. It's whether you're making important decisions based on feedback you can trust.

How should you assess your exam readiness?

My advice is to use AI as a supporting resource, not as your examiner.

Rather than relying on its percentage, use it to highlight unclear explanations, identify possible omissions or question the relevance of your answer. Always verify its feedback against authoritative materials.

For a more reliable assessment of your performance:

• Use official past papers, suggested solutions and examiner reports.
• Study the marking guidance and understand where marks are awarded.
• Practise applying technical knowledge to unfamiliar scenarios.
• Seek feedback from experienced lecturers, tutors or markers familiar with your examination.
• Track your performance across several timed practice questions rather than relying on one result.

Even experienced tutors cannot guarantee the mark an official examiner will award. But assessment grounded in the relevant standards provides a much stronger basis for preparing effectively.

My final thought

AI has enormous potential to support professional education. But we must not confuse the speed of feedback with the quality of assessment. A detailed response isn't necessarily an accurate assessment. And a confidently presented percentage isn't proof of exam readiness.

The question isn't whether AI thinks you've passed. It's whether you can demonstrate the knowledge, application and judgement your examiner is looking for.

Those are two very different things.