The interesting part of this project wasn't figuring out how to use AI to automate Finance work. It was figuring out which parts should never be left to AI in the first place.
At Honeycomb, Finance spent roughly a week every month reconciling contracts against ARR. On the surface, this looked like a pretty obvious automation opportunity. Contracts came in, Finance reviewed them, someone recalculated the numbers, compared them against what was in the system, found the discrepancies, fixed what needed fixing, and eventually closed the month.
So the original question was essentially: Can we use AI to make this faster?
Yes. But that turned out to be the least interesting question.
The actual problem was that the truth required to calculate ARR didn't live cleanly in any one system. Some of it was in Salesforce. Some was in Snowflake. Some was buried in the actual contract PDFs. Some existed in spreadsheets. And some of the rules existed mostly because experienced Finance people knew them.
Before I could automate the process, I had to reconstruct how the process actually worked.
That changed the project from "use AI to close faster" into something much more useful: encode the things we know, then ask people to deal only with the things that genuinely require judgment.
I started by rebuilding the decision
I pulled the contract records from Salesforce, the corresponding fields from Snowflake, and the source PDFs themselves. Then I started comparing them.
Not just field against field. I wanted to understand how Finance got from the contract someone signed to the number the company ultimately recognized.
That meant sitting with Finance and working through the ugly cases. What happens with this SKU? What happens when the contract has this discount? Why does this field say one thing while Finance calculates another? What makes this contract normal but that one an exception? Where is that rule documented?
And, occasionally, the answer was basically: it isn't. We just know.
Those conversations were the product discovery.
I mapped the rules, formulas, mappings, exceptions, and dependencies until I could reproduce the decision Finance was making manually. I wasn't trying to automate anything yet. I was trying to make the logic legible.
Because if I couldn't explain how the correct answer was produced, I had no business automating it.
Then I separated interpretation from calculation
This was probably the most important technical decision in the entire system.
Contracts are messy. The same concept can appear in different places, under different language, with qualifications buried somewhere else in the document. Understanding what a contract actually says requires interpretation.
That is useful work for an LLM.
ARR and iACV calculations are different. Once the relevant facts are known, Finance needs an exact answer. The same inputs should produce the same result every time.
So I didn't ask Claude to do both.
I used AI to interpret and structure information from the contract, compare that evidence against what existed in our systems, and identify where something didn't line up. Then I moved the actual financial calculations into deterministic logic.
If the inputs are A, B, and C, the result should be X every single time.
There is no prize for using an LLM where a formula works better.
That became one of the basic architectural rules of the system: use AI where the work requires interpretation; use code where the work requires certainty.
Once I understood the work, I built the workflow
I built the first version myself.
I wrote the SQL, prompts, Claude skills, formulas, scripts, schemas, mappings, workflow logic, and executable workflows using Claude Code, Python, JavaScript, SQL, and VS Code.
The workflow started with the source contract PDF in Salesforce. The system extracted the relevant contract terms into a structured schema, pulled the corresponding data from Snowflake, and compared the two.
From there, it had to answer a series of increasingly important questions.
Do the contract and the system agree?
If they don't, what is different?
Does an existing business rule explain the difference?
Can we calculate the correct financial value deterministically?
Or is there something about this contract that actually requires a person?
For normal contracts, the system could extract the terms, reconcile them against the system-of-record data, apply the ARR and iACV rules, and resolve the reconciliation without Finance manually rebuilding the calculation.
When something didn't line up, the system surfaced the discrepancy, the underlying evidence, and the proposed resolution in a Claude-generated HTML interface. Finance could review each difference and approve or reject it inline.
The goal wasn't to remove Finance from the process. It was to stop asking Finance to spend its time proving things the system could already prove.
89% meant we knew when not to automate
The first implementation automatically reconciled 89% of contracts end to end.
That was the automation rate, not the accuracy rate.
For those contracts, the system could extract the relevant terms, compare them against Snowflake, apply deterministic ARR and iACV rules, and resolve the reconciliation without Finance rebuilding it manually.
The remaining 11% weren't failed automations. They were contracts with things like unusual discounts, custom POC work, or terms that fell outside normal SKU behavior.
I could have kept pushing for a higher automation rate. I didn't.
Those were exactly the cases where Finance judgment was valuable.
This distinction matters because automation metrics can create some pretty bad incentives. If the goal becomes "automate as much as possible," eventually you start asking the system to make decisions it has no business making just to move the number.
My goal wasn't 100% automation.
It was 100% appropriate handling.
Human review was part of the system, not a fallback
Once I stopped treating human involvement as a failure state, the exception workflow became much easier to design.
Unresolved contracts were collected into an autogenerated spreadsheet, and Finance received a Slack notification when they were ready for review.
The spreadsheet is probably the least technologically interesting part of this entire project.
It was also exactly what Finance wanted.
They already had a way they liked to work. I didn't need to introduce another interface, another dashboard, or another place they had to remember to check just because I had built an AI system.
That is one of the things I think we get wrong when we design automation. We spend so much time thinking about what the technology can do that we accidentally make the humans reorganize themselves around it.
The system should absorb as much of that complexity as possible.
The exceptions became part of the product
The other useful thing about that 11% was that every exception told me something.
Maybe it was a genuinely unusual commercial arrangement that should always require Finance judgment. Maybe it exposed a repeatable business rule we hadn't encoded yet. Maybe the underlying data model was missing something. Maybe the extraction was wrong. Maybe my logic was wrong.
Those are very different problems, and I wanted the system to distinguish between them.
Finance reviewed the exceptions. Their decisions became evidence. When the same kind of decision appeared repeatedly, I could determine whether it should become a rule. If it did, the system handled that case the next time.
So the exception path wasn't sitting outside the automation.
It was how the automation learned where its boundaries should move.
I wasn't trying to get Claude to answer more questions over time. I was trying to make the line between automated work and human judgment increasingly precise.
Before I trusted it, I evaluated it
Getting the workflow to run was one thing. Asking Finance to trust it was another.
I built evaluation sets using historical Finance decisions as the reference and compared the system's behavior against them. I reviewed false positives and false negatives, looked at where classifications failed, and tested whether the system was correctly distinguishing normal contracts from genuine exceptions.
If the system said a contract could be reconciled automatically and Finance disagreed, I needed to know why.
If the system sent something to Finance that could have been resolved automatically, I wanted to know that too.
And if Claude interpreted something differently than a person did, I didn't assume either one was automatically right. I went back to the source evidence.
That process let me improve the system, but it also made the system explainable.
Instead of saying, "Claude thinks this contract is fine," I could show what the contract said, what Snowflake said, which rule applied, how the calculation was produced, and whether a person had been involved.
For a financial workflow, that isn't extra documentation.
That's part of the product.
The interface mattered as much as the architecture
There was plenty happening underneath this workflow: Salesforce PDFs, Snowflake, structured extraction, Claude skills, MCP-connected tools, deterministic Finance rules, evaluation logic, human approvals, autogenerated spreadsheets, and Slack notifications.
Finance did not need to care about most of that.
They needed to know:
What did the contract say? What does the system say? Why are they different? What does the system recommend? Do I agree? What actually needs my attention?
That was the job of the review interface.
I think this is particularly important with AI products because technical sophistication can hide a lot of bad product decisions. A system can have an impressive architecture and still make someone's job harder.
The complexity belongs underneath.
The person using the product should get the decision, the evidence, and a clear way to intervene.
The final system was deliberately boring in places
The finished workflow used AI, deterministic code, formulas, human judgment, spreadsheets, Slack, and the systems Finance already used.
That mixture was intentional.
I wasn't trying to build the most agentic Finance system I could. I was trying to remove roughly a week of unnecessary work without introducing a new category of financial risk.
Those are very different objectives.
And I think that distinction becomes more important as models get better.
The question isn't simply whether AI is capable of doing a task anymore. Increasingly, it is.
The harder product question is what authority we should give it, what evidence we need before we trust it, what should remain deterministic, and where human judgment is valuable enough that removing it would actually make the system worse.
What I would build again
The system ultimately changed Finance's role in reconciliation.
Instead of manually reconstructing every contract to determine whether something was wrong, the system handled the contracts it could prove and brought Finance the ones that actually deserved their attention.
89% of contracts were reconciled automatically end to end. The remaining 11% went to people on purpose.
But 89% isn't really the part of this project I care about most.
The part I would carry into the next build is the architecture of the decision.
Let AI interpret what is messy.
Use deterministic systems for what must be exact.
Give people the cases where judgment actually adds value.
Treat their decisions as evidence.
And don't make someone learn a new tool when a spreadsheet genuinely works just fine.
AI didn't make this system trustworthy.
Knowing exactly where AI should stop did.
Read the full case study: Rebuilding finance close with AI at Honeycomb