PwC puts guardrails on AI

• Key insight: Uncover how PwC is using agent harnesses to move toward artificial general intelligence.
• What's at stake: Corporate boards scrambling to oversee rapid AI integration amid mounting cybersecurity risks.
• Expert quote: "We're not there yet." — Shawn Panson, PwC US assurance transformation leader

PricewaterhouseCoopers has been ramping up its use of artificial intelligence in recent years, but as worries increase about the possible dangers, the Big Four Firm is also making sure to control the technology, even as it seems to be moving toward the threshold of artificial general intelligence.

"We have very strong governance before we release anything," said PwC US assurance transformation leader Shawn Panson. "Understanding all the model discussions, we're using them for very discrete and intentional purposes."

Panson acknowledged that the technology is evolving quickly: "The technology is evolving incredibly fast, and like others we're obviously trying to keep up with it and make sure we're using it responsibly, but also using it to the extent of its capabilities, making sure we get the most out of the potential."

Corporate directors survey

On Wednesday, PwC released its annual corporate directors survey, which found that 71% of the 561 directors surveyed say their boards need to strengthen their AI skills to provide more effective oversight, more than twice the next-highest response. Some 69% surveyed identify cybersecurity, data privacy or intellectual property risks as their top AI-related concern, ahead of significant investment without clear returns (44%) and overreliance on AI outputs leading to weakened human judgment (35%). 

PwC is finding many uses for AI when doing work for its clients. "One of the great things AI has given us the ability to do is to ingest client data and trial balances," said Panson. "It's always been one of the biggest challenges in the profession. Our clients have hundreds of different ERP systems all customized in a different way, so there's always been this drive toward a standard data model."

"If you think about what an audit is, it's a sort of end-to-end process of  testing the client's data," he continued. "We've been able to build a harness upfront that's ingesting the data, putting it in a format that works for us, and then feeding it downstream. That may be feeding into our controls testing process, feeding directly into our substantive testing process, and we're able to ingest that data once and use it multiple times rather than having to ask the client for very similar pieces of information multiple times."

Artificial general intelligence

PwC hasn't yet achieved artificial general intelligence, and many AI experts disagree on the meaning of AGI.

"We're not there yet," said Panson. "But as we move there, we're seeing more and more possibilities. When we do a task once — let's say the intelligence does it incorrectly or the harness does it incorrectly — we have our humans still checking everything. And then between the human annotating back to the agent, and then the agent giving us information, we can actually continue to learn and make the agents better in real time, both through human interaction with them and through the agents having a loop back where they can actually have some level of self-learning to continue to improve themselves over time."

The agent harness can be programmed with contexts and reach into different places to get information. "The LLM is just the raw brain, but the harness is the scaffolding," said Panson. "It allows you to go in different directions and places to pull documents. It really allows you to pull together a different user experience that then can leverage the brain. The harness is sort of the body and the nervous system."

So far, he hasn't seen any surprising behavior exhibited by the LLMs. "I don't think they're doing anything surprising, but we continue to get excited by the performance of the harness with the model," said Panson. "We continue to see improved performance, and that's part of our governance is measuring the model, the model effectiveness and the model accuracy."

PwC has been working with OpenAI and Microsoft and has access to large language models from the other major AI vendors as well. 

"Each one of those tools has strengths and weaknesses of all the major companies, and we're using all of them," said Panson. "But what the AI engineers are so good at is knowing which model performs better for which kinds of tasks, and ensuring that's what we're calling on with our experience. We're not beholden to any one of the large language models, but we have access to all of them, both on an enterprise level and as well as our individuals. We are training up all our employees, and they have they have access through our secure environment to use those tools day to day for their own administrative tasks or whatever else they might need to do for very discrete, specific client examples."

Introductory bullet points created by AI with editorial review