Zoptimizer: testing improvements to a bookkeeping agent

A workflow for finding repeated work in Zaitgeist and testing changes against both processing time and bookkeeping quality.

Contribution
AI system design · LLM benchmarking · Agent optimization
People
  • Baran GüneyselInteraction Designer
  • Julien EudineML Engineer

Understanding where time goes

Zoptimizer studies how Zaitgeist completes bookkeeping tasks. It examines the steps taken during a run, identifies work that may be avoidable, and tests a proposed change against the original version.

The aim is to reduce waiting without leaving more bookkeeping unfinished. Timing and quality are assessed together, using the same measurement rules from one comparison to the next.

Investigating repeated work

For example, an agent might look up the same information several times while processing a receipt. Zoptimizer flags that repetition so a comparison can establish whether one lookup would be enough.

It also looks for checks performed one after another that might be grouped. Some steps depend on earlier answers, so grouping them is a possibility to investigate rather than an automatic improvement. Records of earlier attempts preserve what was tried and why it helped or failed.

Testing the change against the same work

The original and modified versions process the same receipt batch in temporary books, separate from live company records. The comparison examines how long they take and what they complete.

A version that finishes sooner but leaves more receipts without matching entries fails that quality check. Duplicate entries and processing failures are also compared. Differences in the books are reviewed against the receipts to assess whether a changed entry is better, worse, or equivalent.

A successful comparison only covers the situations in that receipt batch. It cannot establish how the agent will handle accounting questions the test did not include. Proposed changes remain subject to review before adoption. Any claimed time saving needs repeated tests and a review of the books.