Accounting firm tech stacks may be leaking more information than leaders realize, including sensitive client data, due to unintentionally agreeing to innocent-sounding terms and conditions that, in truth, give vendors permission to access and use it how they see fit, such as training their own AI models. As a result, many firms are likely in violation of any number of rules, regulations and standards regarding data management and financial privacy without even realizing it.
Charlie Barmore, a tax practitioner who specializes in AI implementations, said this issue encompasses both small startup vendors and large widely used platforms. While the size differences are obvious, both may potentially be accessing sensitive data the practitioner would rather they not. This is because both may potentially have inserted legal language in their governing documents — whether that's an end user license agreement, a data partnership agreement, a privacy policy, a terms of service, or even the contractual language itself — that give them permission to do so. And this is not even a new issue; such language has been present for years for certain vendors.
"If you go and read their terms, they're not exactly favorable. Some of the terms have been like that for several years," he said.
While there is great variation between how vendors word their respective policies, however, there are certain common terms, phrases or clauses that should raise eyebrows. Barmore said that the first thing though, before anything else, is to understand how the vendor defines "customer data." This allows a reader to then understand what their agreements actually encompass.
From there, he said, people should look for words such as "improve," "enhance" or "develop," particular if they're placed near descriptions of services or products. Other red flag terms include "share," "disclose," or "make available." Another thing leaders should look for is whether there's any language regarding data licensing and, if so, how long this license is for and what it allows. Sometimes, he said, firms accidentally agree to a perpetual irrevocable license that remains in effect even when the business relationship is severed.
"Those are some you know things I would definitely try to CTRL-F and see the surrounding language around those kinds of key terms," he said.
Viral Dhimar, chief information security officer at top 50 firm Sikich, is also aware of how certain vendors will try to sneakily access sensitive data. Part of the firm's vendor review process is looking at things like privacy policies, API documentation and trust centers to look for phrases that go beyond what the contract stipulates. Terms and phrases that might suggest this include "Develop new features," "Model improvements," "Training data," "Telemetry," "Anonymized data," and "Improve product and services."
"If in a contract or other areas mentioned above state that the vendor could 'use customer data that is anonymized to improve product and services,' Sikich and other accounting firms should ask exactly how the data is being used to improve said product and services," he said in an email.
Brian Tankersley, an accounting technology advisor with K-2 Enterprises, stressed overall how important it is to have someone qualified actually read the documents that come with a new solution and be able to compare it with other similar products.
"I would say that I think somebody needs to read the whole privacy policy and the whole terms of service. It doesn't have to be everybody. But somebody that is an expert needs to read these. I think you need to run them through something similar to my AI thing, so that you can compare the things apples to apples to each other," he said.
These documents can be long, dense and complex. Sometimes, even after looking for key phrases and terms, people can still be confused. Tankersley noted that this is an area where AI might be able to help. When evaluating solutions himself, he copies the relevant information into a document, uploads it to an LLM, and has the model analyze the policy in plain language. He, himself, uses the below prompt:
You are an expert in privacy and a gifted technical writer which takes complex documents like privacy policies and clearly summarizes them for professionals. When I input a privacy policy, you should respond with a summary document as follows:
Format for Privacy Policy Summaries
- Product Name
- Publisher/service provider name
- Policy document name:
- 300 word summary of privacy policy
- Key phrases from privacy policy regarding information sharing with others
Listing of explicit third parties/subprocessors with whom information is shared, in a table, with columns as follows:
- Name of subprocessor
- Purpose of information sharing
- Kind of information shared
- What kind of information can users opt out of being shared
- Directions and/or links for how to opt out of information sharing
Subprocessors and their subprocessors
All three noted that a major complicating factor in vetting solutions for privacy is how integrated a lot of software is with subprocessors, loosely
This means that even if the actual solution a firm uses checks out, the third parties that vendor works with may not. Therefore, it is important to not only look at the governing policies of the direct vendor but also understand what other companies work with that vendor. Similarly, it is important to examine whatever platforms a solution has integrated with as well, since data could flow through there as well.
"This is what's making it very challenging, because these solutions may be using a third party to then produce this specific output or implement this certain feature. So we still do need to vet the software that we're using and any of its known sub processors or any of the vendors that it uses," said Barmore.
Such evaluations are part of the standard evaluation process at Sikich, according to Dhimar.
"As part of vendor management at Sikich, we evaluate AI/LLM models and other technologies that access information. We ask what data the vendor will be accessing, where it is stored, how it will be utilized and what third or fourth party will also potentially process it," he said.
While this might seem simple in principle, Tankersley said it can be much more complicated than people may at first think because even when someone finds out who the subprocessors are (and it is not always disclosed), these subprocessors are not necessarily easy to find and they don't always make their documents readily available, and when they do they're not always clear. And compounding this issue is that even if the subprocessor checks out, there is still a need to check out the subprocessors to the subprocessors.
"Okay, here's the problem. One, you have to go find their terms, and two, they're not always explicitly disclosed, so you have an opaque thing. … One of the things they're required to by law is disclose who those sub processors are. Okay, but the problem is the sub processors have sub processors," he said.
The consequences for ignoring subprocessors can be severe. He talked about someone who used a payroll solution that used a third party subprocessor to act as a bank feed aggregator, as many platforms do. The solution's documents mentioned its own terms and conditions, but not the subprocessors; instead, they simply said that by agreeing to their rules, they also agreed to the subprocessor's rules without explaining what exactly those rules were. If you went to the subprocessor's actual site and looked up their own terms and conditions, the user would have seen that they reserve the right to share user data with other third parties, such as credit reporting bureaus like Equifax.
"I've had CPAs that had clients that went up to them where they were using a payroll solution that was talking to Equifax, and they applied for something, and the bank already knew, because of Equifax, that they made a million dollars. And [the client] said 'My accounting firm does this. That's the only place it could have possibly leaked out.' And it [became] a relationship issue for that firm. And so part of the challenge with this is how do we work in this world where everything's integrated, and we need to make sure everybody keeps their mouth shut?" he said.
Complicating things even further, according to Tankersley, is the fact that terms and conditions can change. So while a subprocessors may have had stringent privacy controls one day, this could change dramatically the next and unless someone has been routinely checking all of these documents for updates, the firm would not even know since they don't have a direct relationship with them. This is especially risky in light of the fact that major AI providers like OpenAI and Anthropic – neither exactly famous for their data privacy – are increasingly serving as subprocessors for vendors who've licensed their models for their own products, as both have already changed their privacy policies several times.
"Most of the terms of most of the terms of service say things like we can amend this and change it as we'd like to. … Now think about this for a minute. You have Claude. You've vetted Claude. You've embedded Claude into your workflows, and suddenly now they're unilaterally changing this in the middle of your workflow after you've implemented it," he said.
And this is even assuming the firm knows they're using Claude; they may not even realize that the solution they bought licenses its models from Anthropic and so they're basically using Claude with different branding.
Vendor vetting
This gets into another aspect of vetting solutions for privacy: asking pointed, specific questions at vendor meetings, including about subprocessors. Barmore said he asks specifically whether or not there is a training clause, and if it is to have them point out the specific line items in the terms to show him how it works. He also said it is important to ask how the vendor defines customer data and how they use it, once again asking that they cite the specific terms rather than just taking their word for it.
"Ask specific questions about what data do y'all collect? What do y'all do with it? Is there training? Where is it stored? Does it use sub-processors? Where are they located, and how is the data used there? It's really best to kind of have the sales rep pull it straight out of the terms, as opposed to relying on what they may say," he said.
Dhimar, from Sikich, said it is a great practice to ask the vendor directly things such as:
- "Can you state in writing that no client data, prompts, output, usage, anonymized or derived data are used to train any model (current or future)?"
- "Can you state in writing that no third party, fourth party, downstream provider, model provider or cloud provider will use the data to train their models?"
- "Where is the data stored after it leaves Sikich network/tenant?"
- "Please identify exactly where in the contract or other binding agreement that prohibits training on our data?"
Tankersley was blunt when it came to what firms should ask their vendors regarding privacy:
"Okay, you want to ask, first, is my data used to train your models? And if it is, then you don't want any part of that action. Secondly, what happens to my private information?" he said.
Technical tests
While documentation and vendor questions can answer a lot, there is always the possibility someone might be outright lying. However, testing this empirically can be difficult given the black box nature of many AI models, as well as the lack of granularity of what data is getting transmitted where.
"It's hard to actually test it, because once it is digested by the software and goes into the cloud, we now have lost access to it. They do the processing. They use it however they have stated they are going to use it and then it's left to us to trust that they're only going to use it the way that they stated in their terms," he said.
Still, firms are not completely helpless in this regard. Dhimar said that buyers should first ask upfront, as part of any vendor management process, for network diagrams, architecture diagrams, data flow diagrams and, of course, a list of third-party, fourth-party and other sub-processors.
Then, as part of network and application monitoring, firms should review unexpected data transfer (the size of the data is a big tell), which cloud regions were used to store or process the data, and API gateway and/or endpoints.
"If data is going to other areas not explicitly identified in the diagrams or documentation listed above, it may indicate a breach of contract," he said.
But how does a firm know whether sensitive data has already left the company and been used to train an AI model? Provided one knows which model may have been trained, Tankersley said someone could ask it to describe things about the firm or the client that, in principle, it should not know. If it begins spitting out sensitive information that no one else should have, then it is likely it was trained on the firm's data.
"Do a prompt like that inside one of those things and see what it figures out about you. What does it know about me from this? … I think in general one of the things people need to be very careful about is what they connect to [AI models] because once you let them in your email, they're in your email," he said.
Regulatory risk
Considering how many products take user data to train their AIs, and considering how many accounting solutions now use AI, and considering how many of those AI models are run through third party subprocessors, it stands to reason that many firms are unknowingly in violation of rules and regulations from a variety of regulatory bodies like the IRS or SEC which mandate client data remain private.
Barmore said a problem, though, is that accountants operate under so many rules and regulations that have not fully adapted to the new technology and so they're trying to fit new practices into old models, which creates a regulatory minefield. While it is unlikely they will realize an individual firm is in violation when the firm itself isn't even aware, Barmore said this should not be confused for never being found out, as any number of things could expose them.
"I think the way regulators would find out was a breach. If there's a security breach in some of these models, and it comes to light, or there's a lawsuit. So, I wouldn't advise operating under that assumption that someone may never find out because I don't think that's the responsible way to go about it," he said.
Tankersley agreed, but at the same time said that this is not a new issue brought out by AI. It is likely any given firm has at least some data leaking somewhere. He pointed to a firm where an intern was looking at health-related audit evidence and they took a picture of a document with their phone, which then automatically went into iCloud, which then counted as a HIPAA violation as iCloud "does not do any of the regulatory stuff that needs to be done."
"And so the problem I think is, first and foremost, people have got to get their house in order with respect to their staff usage," he said.
Besides analyzing documentation, questioning vendors and performing technical monitoring, Barmore also said it's important to find whether or not a solution has a way to opt out of providing training data. Sometimes it's as simple as clicking a toggle in preferences. Still, he conceded it was an imperfect solution since it does not address actual data storage and retention on the part of the vendor.
Tankersley, meanwhile, suggested seeing which companies work directly with the federal government, which tends to carry stringent data control requirements on their own; while doing this does not completely eliminate risk, he said it should lower it. But more than anything else, according to Tankersley, is making sure you actually read what you agree to, something he felt far too few people do today.
"I think generally the bigger issue here is that we have just said 'yeah, I'm going to click on okay and accept that.'You know the problem is if that thing said 'I give somebody title to my left arm and my firstborn child' some people would still just click on OK," he said.
