These assessments are said to be performed by "Renata," an AI EQA assessor trained by the firm's own assessors on completed engagements and programmed to work on a quality assessment the way any quality specialist on their team would, and "Tomasz," an AI QA lead responsible for pre-clearance review by reading the whole package against the QAM quality criteria before issuance which serves to keep ratings consistent from one engagement to the next.
"None of these are human," said Cherry Hill Advisory CEO Mike Levy in an email. "[But] we have them following a very similar workflow to what our team uses, like updating our project management boards, and our primary engagement director can interact via chat, etc. … These are not humans, but we have built humans into the overall EQA process since the human in the loop is important and also required by the global internal audit standards."
The engagement, despite being led by AI, follows similar steps as any other. During the scope and charter stage, independence is confirmed, stakeholders are identified, and document requests are issued and tracked. During the evidence-mapping phase, charters, plans, workpapers and QAIP evidence are read and mapped to every QAM standard with citations. During the interview and survey step, structured interview documents and survey forms are provided to the team, management and committee. During the opinion and report period, clients get a conformance opinion with an included maturity rating and prioritized commendations.
Every rating, conclusion and final opinion is reviewed and signed by a Certified Internal Auditor in compliance with Global Internal Audit Standards (Global Internal Audit Standards and QAM are issued by the Institute of Internal Auditors), so while the agents may assemble and document, the judgment still ultimately rests with a human. The engagement file records who reviewed each step and where the reviewer agreed or disagreed.
"Every single step in the agentic EQA needs to be reviewed and confirmed by a qualified human assessor," said Levy. "The humans can do the review and sign off, and also leave review notes for the agentic assessor, who then can process those review notes and make all the changes. We also would rely on the human team to have some of the interactions with our client and do the reporting at the end to the audit committee."
Ultimately, clients receive a conformance report; a committee-ready deck carrying the opinion, the evidence and the action plan; a badge and plaque issued on a full or generally conforms result; and a portal listing for a verifiable credential record with issue and expiry dates.
Levy said the AI-driven process is about 50% percent faster than a typical engagement, depending on circumstances. This arrangement also allows the firm to pass what Levy said were significant savings onto the customer.
"On average we are seeing these assessments cost 30% to 50% less than our traditional assessments," he said. "My goal is to make this accessible to every function, regardless of budget, since there are many functions that can't do this work because of budgetary constraints today."
