Advisory and audit solutions provider
Specifically, the new Audit Testing Agent automates the process of matching client evidence to samples, extracting and validating key data from documents, and annotating and documenting test results. The Audit Testing Agent supports a wide variety of document-based audit tests, including revenue cut-off, expense verification, unrecorded liability testing, and fixed asset additions.
It is being rolled out as an enhanced feature of Fieldguide's existing audit platform, meaning current Fieldguide customers can now access its capabilities. Fieldguide CEO Jin Chang, in an interview, described the new offering as a true end-to-end audit solution that encompasses the entire engagement lifecycle. He contrasted it with similar products in the market, which he said are more like point solutions that handle only one or two steps in the process or are meant for very specific applications like invoice testing.
"When we talk about agents, we think of agents as a more holistic, multistep workflow approach," he said. "Our argument is actually that many solutions in the market are not quite agentic. They're more [like the] AI workflows that Fieldguide has already been building."
He noted that many solutions will surface discrepancies and possibly make suggestions for manual adjustment. In contrast, he said, Fieldguide's new AI will go beyond these steps and do things like suggest follow-up questions to clients, draft the communication, evaluate the client response, perform a quality check on the new evidence provided by them, and "connect the dots back to what the auditor is testing for."
"By the time the response gets back to the audit team, Fieldguide AI has already pre-tested for quality after evidence and responses come back [to them]. Our agents will test again, document the results, and ultimately the results that are documented flow all the way through to the end financial statement reports," said Chang.
He noted that this approach still retains a human in the loop philosophy, which means it's not actually initiating the client communications with no supervision. While theoretically it could act much more on its own, he said CPA firms are not yet comfortable with that level of independent action from their tools. He contrasted this with other industries, such as software development, where agents are being built with a significantly higher degree of autonomy. Still, this does not mean the AI sits idle waiting for the human to interact with it. Even with this more controlled approach, the agents are still performing some tasks independently.
"Fieldguide field agents can be autonomous at a very extreme end," said Chang. "However we need to make sure to meet CPA firms where they are in their AI transformation journey. … What we found is that current levels of comfort in the industry [necessitates] a human in the loop approach where our agents are suggesting next steps and doing proactive analysis. For example when the client uploads evidence, our agents are [performing this analysis without] waiting for the auditor to check. I would say we are about halfway in the journey of more full autonomy, mostly because the level of comfort in the industry is at this current place."
Understanding that audit methodologies can vary greatly, the solution sports a high level of customization at multiple levels. This includes the ability for users to set their own materiality thresholds along with their own risk preferences and other best practices. Once set, the AI will use reinforcement learning to better understand how the auditor does things and match itself to their habits.
"We have customization at every level: firm, practice, partner and down to per engagement preferences too, because we found that even with the same partner, two different clients, he or she may prefer a different way of doing things too," said Chang. "So what we have incorporated is reinforcement learning at the engagement level, at the audit level, so that the client specific preferences continue on a year to year basis."
Chang said he is "very confident" in the quality of the AI's outputs, saying they had to design with quality in mind: while consistent accuracy is important for everyone, it is "non-negotiable" for audit professionals. This quality is at least partially driven by what he said was a proprietary evaluation framework that generally involves a series of specialized LLMs monitoring the outputs of the primary LLM for errors and exceptions it may have missed. Using this framework as a check on accuracy, Chang said the AI has been able to not only perform tests much faster than humans, but it has also been able to find errors that human teams made in previous audits.
"Fieldguide's goal is to enhance the quality of audits. We want to help CPA firm partners sleep better at night too, knowing that their audit quality is market leading, not just [producing] efficiencies at the margins. We take a lot of pride in the quality of our AI outputs. We actually would love to see other AI players in the space care more about quality, not just speed. We think that's just better for the market," he said.
While some developers take the approach of having the LLM simply interpret and communicate the calculations made by more deterministic AIs, Fieldguide has the LLMs themselves doing the work, with the specific task matched to the model best equipped to perform it. By giving them access to the right tools, said Chang, LLMs can carry out a wide variety of tasks on their own.
"Based on our evaluations, certain LLMs tend to be better at math and other very deterministic use cases, whereas other LLMs are better at creativity or understanding documents or images and so on. I will note that anyone who makes blanket statements around LLMs not being good at one particular thing, I would argue, is not doing a proper evaluation across other LLMs," he said.
In general, client data is encrypted and stored in Fieldguide's secure AWS environment. In some cases, when working with very large firms, they will work through their own cloud infrastructure instead, but Chang noted this is more of a premium enterprise service for international firms with global mandates.
Chang added that Fieldguide is ISO 27001 certified, completes annual SOC 2 reports, and will soon be ISO 42001 certified as well.
He estimated that, for a mid-sized firm of 100 professionals, implementation time would be between three to four weeks; for a larger firm it might be between three to nine months, depending on the scale of the rollout.
Pricing is generally per-engagement, as the intention is to help CPA firms be more efficient. He argued that per-seat pricing disincentivizes efficiency, as the vendor makes more money the more people use the product. The purpose of this new solution, said Chang, is to enable firms to grow more without having to hire more, a goal that would be at odds with a per-seat pricing plan.
"A lot of CPA firms who've been using our generative AI features the last several years are now reaching a point where they could use another step change in human productivity and quality. … The firms upgrading to our Field Agent solution can grow the top line without necessarily growing headcount one to one," he said. "Our goal is to help CPA firms create nonlinear growth with revenue compared to headcount."
