How Law Firms Should Security-Review an AI Vendor
A partner comes back from a conference sold on an AI timekeeping tool and wants it live by month-end. Before that happens, somebody at the firm has to decide whether software that watches lawyers work is safe. Usually that somebody is you.
We build Ajax, an AI timekeeping tool for law firms, so we sit on the receiving end of these reviews most weeks. You learn a lot from that seat. The firms that run a sharp review tend to ask the same handful of questions, in roughly the same order, and they keep pushing until the answers get specific. Here's the list, along with what a solid answer sounds like.
Map the data flow first
Ask the vendor to walk you through the whole path. What gets captured? Where does it travel? Where does it come to rest, and who at the company can touch it along the way?
For a timekeeping tool, the straight version of that answer sounds something like this: software on the lawyer's desktop reads on-screen work and pulls in signals from email, calendar, and phone systems; all of that goes to the vendor's cloud, gets processed into draft entries, gets stored, and eventually gets deleted. A strong answer names the systems, names the cloud, and names the specific access controls.
A complicated answer is fine. What you're testing is whether the vendor can draw the picture in one conversation, without checking with engineering and getting back to you next week. A vendor who can't say where your data physically sits, or who internally can see it, either hasn't thought about it or would rather you didn't ask. If your firm operates under data residency rules, this is also the moment to ask where each class of data lives. For Canadian firms especially, that answer matters.
Ask what gets deleted, and when
Once you know where the data goes, ask how long each class of it lives. There's usually more than one class: raw capture, the processed entries built from it, and the finished time in your billing system are different things with different lifespans.
The answer you want is deletion that runs automatically, by default, with nobody at the firm having to file a request. Be wary of any version of "we hold onto it until you ask us to delete it." Data that sits around indefinitely can be subpoenaed, breached, or mishandled long after everyone's forgotten it's there. At Ajax, captured data is deleted automatically on a rolling basis, and a security review is the right place to get the specifics for each data class in writing.
Does any model train on our data?
Ask this one plainly: does any AI model, yours or a subprocessor's, train on our firm's data?
The answer you need is a contractual no. A friendly assurance on a call doesn't count. Most AI tools route data through third-party model providers, so what really matters is what those providers are allowed to do with it, and "allowed" is a question about contracts. Look for zero-data-retention agreements with every subprocessor and an explicit ban on training. Ajax's subprocessors are contractually barred from training on firm data and operate under zero-data-retention terms.
A vendor who hedges here, or points you at a general privacy policy instead of a specific commitment, is telling you something. Keep pushing until you get a clear yes or no.
Encryption, access, and identity
The baseline questions still earn their place. Is data encrypted in transit and at rest? Can lawyers sign in through your identity provider? Ajax works with the major ones, including Okta, Microsoft Entra, and Google, which means disabling a departing associate's firm account kills their tool access in the same motion.
Ask about the inside of the vendor's shop, too. Who on their team can see customer data, under what conditions, and does that access get logged? A good vendor has a short, specific answer ready. Most of what people mean when they ask whether AI timekeeping is safe for a law firm comes down to these controls, and you can check them one by one.
Privacy inside the firm counts too
Most security reviews look outward at the vendor. If the tool watches how lawyers work, though, someone at your firm should also be asking who inside the firm can see that activity.
With Ajax, draft entries and captured activity stay private to the individual timekeeper until that person chooses to release entries. Management sees rolled-up reporting - capacity, unreleased hours, firm-wide totals - and never another person's screen or unreleased drafts. We designed it that way on purpose. Attorneys actually adopt the tool instead of quietly disabling it, and far fewer people can see sensitive activity in the first place. Associates ask about this constantly, which is why we wrote a whole piece on whether a boss can see what you're doing.
Get it in writing, and have IT read it
Verbal answers on a sales call are a starting point. Ask for the security documentation and the vendor's third-party audit status in writing, then hand the packet to whoever owns security at your firm, whether that's your IT director or your outside MSP. They read these documents for a living, and they'll catch gaps that a partner skimming on a Friday afternoon never will.
Firms skip this step when they're in a hurry, and it's the step that protects you if something goes sideways later. A vendor who's confident in their security posture hands the documents over without friction. If you have to chase them, that's a bad sign.
You don't have to be a security engineer to run this review. Patience gets you most of the way: keep asking until the answers get specific, and pay attention when they never do.
FAQ
What should a law firm ask an AI vendor before buying?
Ask the vendor to map the full data flow: what's captured, where it's stored, who can access it, how long it's kept, and whether deletion runs automatically. From there, confirm that no AI subprocessor trains on firm data (the answer should be a contractual no), check for encryption in transit and at rest, confirm single sign-on with your identity provider, and request the security documentation and audit status in writing for your IT team or MSP to review.
Does Ajax train AI models on firm data?
No. Ajax's AI subprocessors are contractually prohibited from training on firm data and operate under zero-data-retention agreements, so your firm's captured work isn't used to train anyone's model.
Who can see what Ajax captures?
Only the individual timekeeper sees their own captured activity and draft entries, and only until they release those entries. Firm management sees rolled-up reporting such as capacity and unreleased hours, never anyone's screen or unreleased drafts.
How long does Ajax keep captured data?
Captured data is deleted automatically on a rolling basis, without anyone needing to request it. The specific retention windows for each class of data are covered in a security review, where we put them in writing for your team.
Running a review and want the answers straight from the people who built the system? Book a demo and bring your hardest questions.
