Harness Lab

Experiments on AI agents in real business work

We publish the setup, the data, the failures and what fixed them. Public and synthetic data only.

Follow on LinkedInDiscuss a process

Method

One protocol for every experiment

01Fix the model, data and task.
02Agree acceptance checks before the first run.
03Change one part of the harness at a time.
04Trace every failure to its cause.
05Publish method, data and limitations.
7 experiments
HX-01In preparation

Can AI build a financial report that reconciles?

Income statements from SEC filings, every figure tied to a tagged fact or flagged.

WE MEASURETie-out rate, silent errors, completion
FinanceExcelReal data
HX-02In preparation

Can agents and people edit one workbook safely?

Every changed cell reconciled to a source and a reason.

WE MEASUREUnreconciled cells, conflicts caught
ExcelReal dataSynthetic data
HX-03Planned

Which part of the harness matters most in Excel?

Each part disabled in turn on the same workbook task.

WE MEASURESuccess rate with each part removed
ExcelSynthetic data
HX-04Planned

When an agent says “done”, is it?

Reported completion compared with independent checks.

WE MEASUREReported versus confirmed completion
FinanceSynthetic data
HX-05Planned

Which errors do agents actually catch?

Synthetic ledgers with planted faults: misposts, sign flips, period mismatches.

WE MEASUREDetection rate by error type, false alarms
FinanceSynthetic data
HX-06Planned

Can AI update a valuation model without breaking it?

An existing model rolled forward one quarter.

WE MEASUREFormula integrity, unlogged assumption changes
FinanceExcelReal data
HX-07Planned

Can AI check contracts without inventing evidence?

Contracts reviewed against a checklist, every finding cited.

WE MEASUREConfirmed findings, missed clauses, unsupported citations
DocumentsSynthetic data