Tally
UK financial oversight, debt triage and intelligence.
Appearance
Dark is the design ground. The light palette is real and meets the same contrast standard, for reading in sunlight. System follows your device.
This build
- Version
- 0.1.0
- Commit
- 7c86a49
- Branch
- main
Changelog
Unreleased
Added
- **The Advisor's voice session in the browser.** The `AdvisorVoice` hook, with SIP.js vendored under `assets/vendor`, registers as the SIP endpoint, places the call to 501, plays the far end, and reports the microphone level. The media socket announces the call's id on `voice:media`, which the page adopts, so the page and the call process share one event topic. `Voice.Live` subscribes to the call's control topic and uses the level to move the stage between listening and thinking, so the server never detects speech a second time. The SIP and TURN settings are read from the environment; the password and the TURN credential have no default. See `docs/plans/voice-console.md`.
- **The Advisor page.** `TallyWeb.AdvisorLive` at `/advisor` is the conversation with Tally: the thread list, the bubbles as a call transcribes, the stage ribbon (listening, thinking, speaking, interrupted) and the per-turn timings. It reads the event contract a call publishes, so the display is live without the page touching the audio. Starting the voice session is the browser hook: the button says **Connecting** while it negotiates and explains blocked audio when the browser refuses to play. See `docs/plans/voice-console.md`.
- **The live call's event contract.** `Tally.Voice.Live` takes an optional `notify: {pubsub, topic}` and publishes stages, transcripts, turn timings and usage through `Tally.Voice.Live.Events`, with `Tally.Voice.Live.Stage` as a pure state machine. The stage is published only when it changes, and a call with no notifier publishes nothing, so the phone path is unchanged. This is what the Advisor page and the call archive will both read. See `docs/plans/voice-console.md`.
- **Full-duplex browser voice, with barge-in and echo cancellation.** A browser over WebRTC, with Asterisk terminating the peer connection and bridging to the same 24 kHz PCM the media socket already uses, holds a full-duplex, interruptible conversation with Tally. The model drives the turns and the barge-in (`activityHandling: START_OF_ACTIVITY_INTERRUPTS` with a high start sensitivity); Tally keeps the figures and the transcript. Proven on IronFox 2026-09-22; needs a TURN server and IronFox's per-site Autoplay allowed. See `docs/plans/voice-console.md` and `docs/decisions.md`.
Removed
- **The uplink denoiser, and Rust from the build.** RNNoise (`native/denoise`, its Rustler NIF and the `rustler` dependency) and the Rust toolchain in both Docker images and the CI base image are gone. Every settled client cleans its own audio, and the handset's real weakness is level, not noise. See `docs/decisions.md` (2026-09-22).
- **Half duplex.** `close_uplink/1`, `maybe_reopen/1` and the per-call `duplex` option are gone; every client runs full duplex.
- **The custom turn detector and the turn-based voice stack.** `Tally.Voice.Live.Listener` and `Tally.Voice.{Server,Ami,Call,Dialogue,Speech,Turn,Wav,Frame,Connection}` are deleted. The model drives the turns.
- **A denoiser on the voice uplink**, so a call works in a used room and not only a quiet one. RNNoise (`nnnoiseless` 0.5.2) runs in a Rustler NIF at `native/denoise`, resampling 24 kHz up to 48 and back down inside the call so the frame contract does not change, and its per-frame voice probability replaces the peak threshold in `Tally.Voice.Live.Listener`. It **fails open**: a filter error hands the raw frame to the model, marks the trace and lets the call continue, because a dropped call is worse than a noisy one. Off by default, switched on with `TALLY_DENOISE=rnnoise`. Measured at 465 µs a 20 ms frame at p50, 740 µs at p99. See `docs/plans/uplink-denoise.md`.
- **A live voice call, on Gemini Live, at 24 kHz.** The owner dials 500 from a phone and talks to Tally about the real position. Asterisk hands the audio to `Tally.Voice.Live` over a WebSocket media channel rather than AudioSocket, because the AudioSocket application hard-codes 8 kHz and discards whatever SIP negotiated. Turn detection is Google's **hybrid VAD**: the server finds the start of speech and Tally says when it ended, so the model answers when the owner stops rather than when the server's patience runs out. The uplink is **closed while Tally is speaking**, because a handset cannot cancel the echo of its own earpiece and the model was transcribing its own voice. The call writes a per-call trace with the pause measured from the line going quiet to the reply starting. See `docs/plans/live-voice.md`; the tools, the transcript into the thread and the numeral audit have not landed yet.
- **`Tally.Voice.Live.Listener`**, the client-side end-of-turn detector: sustained speech to open a turn, 800ms of real silence to close it, Tally's own voice excluded from the noise floor, and the floor allowed to move only on frames that were not speech, so a sentence can never silence itself.
- **`Tally.Voice.Live.Trace`**, one JSON file per call under `:trace_root`, and a log line per turn with the timings and the settings the call ran under. Written because the argument about why the call was slow had been conducted entirely in guesses.
Fixed
- **A failed call could take the media socket down.** `TallyWeb.MediaSocket.handle_in/2` read `state.live` even when the call had never started, so a startup failure crashed the Bandit connection process. The guard `terminate/2` already carried is now in `handle_in/2` too.
- **The half-duplex mute reopened partway through Tally's reply.** The window was keyed to the first downlink frame of a turn and never refreshed, so 600 ms into a long monologue the uplink reopened and every later frame re-closed it, sending `audioStreamEnd` and resetting the listener each time. The handset's own earpiece then leaked back mixed with the owner's voice, which the trace on 2026-09-21 showed as end-of-speech marks inside Tally's turns and `<noise>` where his questions were. The window now runs from the last frame, so the mute holds for the whole turn as the design says.
- **An estimate was read aloud as an exact figure.** The live brief matched on the amount alone, so only a zero could ever be labelled, and a positive estimate reached the model looking exact. This is the rule in `AGENTS.md` that an estimated figure never looks like an exact one. Caught by a test, on a value the code could not previously express.
- **A turn ran for 41 seconds** and then answered a question nobody had been able to ask, because turn detection had been left to the server on a line that never went quiet.
- **The model answered its own echo.** "Yes please" came back transcribed as "Yes, Países" on a real call, because the handset's microphone hears the handset's earpiece and the model was sent both.
- **Downsampling aliased.** The uplink resampler interpolated without filtering, which folds content above the new Nyquist frequency back down, so the module was deleted rather than improved.
- **Silence at both ends of the call.** The phone's audio path drops the first frames after the call is answered, so the message began partway through, and the hangup clipped the last word. A second of silence at each end fixes both. Reported on a real call, both symptoms.
- **The audio is paced, not sent in one burst.** Asterisk queues what the AudioSocket server sends and plays it in real time, so sending a whole reply at once and then a hangup means the dialplan hangs up and discards the queue. The first call that reached this point sent three seconds instantly and the owner heard nothing. Each chunk now waits out its own duration, including the last, before the hangup.
- **Audio is `0x10`, not `0x03`.** Asterisk 22 moved AudioSocket's audio type from `0x03` to `0x10` and gave `0x03` to DTMF, which is received-only. Tally was written from the older protocol, so the first answered call sent the caller's keypad back at Asterisk and the call died two seconds in with nothing said. `0x03` is now decoded as DTMF, and a test pins both kinds.
- **A streamed WAV is read, not discarded.** The Piper server leaves the RIFF and `data` sizes as placeholders and fills them in afterwards, so a `data` chunk whose stated length runs past the end of the buffer is the audio to the end rather than a malformed file. Found on the first real call, where the reply came back empty and the phone said nothing.
Fixed
- **The call drains the line before it listens.** Asterisk sends the channel's audio continuously, and while Tally speaks, the phone's microphone picks up the earpiece and sends it back, so a fourteen-second greeting leaves fourteen seconds queued and the listener reads its own voice, seconds out of date. On the first real conversation that showed up as **"I did not catch that"** after a perfectly good question. The queue is now discarded before listening, and a caller who is not heard is logged with the loudest frame seen, which is the difference between a threshold set too high and a line that is simply dead. Writing the drain also found that `gen_tcp` treats a timeout of zero as infinity.
Changed
- **The self-hosted voice stack is removed, and it goes cloud.** The models held 3.7 GB of a 4 GB GPU shared with Jellyfin, which needs the card to transcode, so the `tally-tts` and `tally-stt` compose services and the `ops/whisper` Dockerfile that built the local whisper are gone, and the embedding model is behind a `gpu` profile so that nothing of Tally's is on the card. Search by meaning is unavailable while the embedding model is down, which is the degradation its configuration was designed for. **Nothing in the application changed**: `Tally.Voice.Speech` only knows a URL, so a provider is configuration. The direction, recorded in `docs/decisions.md`, is a cloud live speech-to-speech model with a strict prompt and the whole context as its knowledge. **The figure rule is unaffected and is the constraint the design turns on**: an end-to-end model generates the words, and a number heard is the hardest kind to catch.
Added
- **A reply may quote the next penalty and the interest.** The conversational context now carries the next charge from the published scale and the interest accrued so far, so asked "what is the penalty" a reply answers it. Its absence was the first real conversation's gap: the call's own greeting said the penalty out loud while the reply, asked the same question, said it was not held. Both are read by the caller and passed in, so `Tally.Threads.Context` stays pure, and `Tally.Penalties.next_charge/2` is the one place the "what lands next" question is answered.
- **The call answers briefly.** The conversation prompt now asks for two or three sentences with nothing said twice, because it is spoken aloud and cannot be skimmed. The first real answers were paragraphs, which is about thirty seconds each.
- **The call holds a conversation.** `Tally.Voice.Dialogue` listens for the caller to stop speaking, transcribes them, answers through the same figure-free reply schema the typed thread uses, speaks the answer with its citations substituted, and writes both turns into the obligation's thread with `transport: :call`. `Tally.Voice.Turn` decides the turn from a root-mean-square threshold, with two waits rather than one so a breath is not a finished question and a silent caller is not left on an open line. Answering is shared with the typed thread through `Tally.Threads.Answer`, and a reply now carries the transport the question arrived on. Landing two of `docs/plans/voice-call.md` is built; a real call has not yet exercised it.
- **Local speech to text.** `ops/whisper/Dockerfile` builds whisper.cpp for this machine's CPU, which is the whole reason the file exists: every published image emits AVX2 instructions, and this Xeon E5-1650 v2 has AVX but no AVX2 and no FMA, so they die with `Illegal instruction`. It runs on the GPU, transcribes 8 kHz telephony audio in about three quarters of a second, and carries Tally's own vocabulary as an initial prompt, without which whisper heard "Tally here" as "Tallyhir". The listening half of the voice call is no longer blocked.
- **VAT is said V-A-T.** `Tally.Obligations.spoken_label/1` separates the letters, because an initialism read as a word is "vat" and every UK accountant says V-A-T. PAYE is deliberately left alone, because it is said "pay". The page still writes "VAT".
- **The call leads with the next penalty.** A call now says the next penalty and the day it lands, then the interest accrued to today, both priced by Elixir against the seeded HMRC schedule and rate. The penalty leads because it is the figure that makes the deadline matter: on the VAT payment that prompted this the interest is £15.85 while the first 3% is £172.26 and lands three days out, so naming only the interest would be reassuringly wrong. Neither line appears before the deadline, and the principal is still never spoken. `Tally.Interest.accrued/2` gained `as_of:` so a delay that is still running can be priced to a date.
- **Tally rings the phone.** `Tally.Voice.place/1` rings the extension through the Asterisk manager, which puts the call into the dialplan where `AudioSocket` hands the audio over, rather than into a Stasis application where Tally would own the media over RTP. `Tally.Voice.Call` then loads the obligation the handshake named, says what it is and how long there is, and hangs up. No amount is spoken, for the reason the push gives, and a failure speaks a short line rather than nothing. Landing one of `docs/plans/voice-call.md` is built
- **The AudioSocket listener.** `Tally.Voice.Frame` is AudioSocket's framing, `Tally.Voice.Server` the TCP listener Asterisk's dialplan hands a call to, and `Tally.Voice.Connection` buffers frames for one call and closes the socket whatever happens. Tally never sees SIP or RTP. A call is spawned unlinked so one crash cannot take the listener, the port is read back rather than assumed, and the listener binds only when `TALLY_VOICE_PORT` is set. Step 2 of `docs/plans/voice-call.md`
- **Hearing and speaking, locally.** `Tally.Voice.Speech` reaches whisper.cpp and Piper over HTTP the way `tally-embed` is reached, and `Tally.Voice.Wav` carries the audio: a synthesiser answering at its own rate is resampled rather than played at the wrong speed, and a failure is a value rather than a raise. No default URL, because a transcript from a different model is meaningless rather than wrong in any detectable way. Step 1 of `docs/plans/voice-call.md`
- **Ask about this obligation.** The thread is on the obligation's page now: the owner asks, the worker answers, and each reply's figures are rendered by Elixir from the citations it snapshotted. A derived figure keeps its dotted underline, so an estimate never reads as exact in a conversation, and a key the model invented says it is not held rather than showing a number. `Tally.Jobs.Workers.ReplyToMessage` answers one owner message on the `:reading` queue, idempotent on the message it answers: the check runs before a model is reached, so a retry neither spends nor writes a second reply, and a model that fails on every route leaves the question standing rather than answering from memory. Steps 5 and 6 of `docs/plans/thread.md`, which is now implemented
- **The conversation facade.** `Tally.Gateway.converse/3` answers a question about an obligation over the single `complete/3` callback: it builds the messages from the context, validates the response against `Tally.Gateway.Reply`, and re-routes to the next model when one writes a figure. The behaviour stays at one callback, and the boundary test's whitelist gained `Reply` and nothing else. Step 4 of `docs/plans/thread.md`
- **The context a reply may quote.** `Tally.Threads.Context.build/3` assembles, without a model or a request, what a conversation knows: the obligation's own figures, the company position the forecast computed, and the kind's glossary entry, each figure keyed so a reply can quote it without writing it. A derived figure is marked as one, so an estimate in a conversation looks like an estimate everywhere else, and a key the model invented does not resolve. Step 3 of `docs/plans/thread.md`
- **The figure-free reply schema.** `Tally.Gateway.Reply` is what a model may say in a conversation: prose with no numerals, the figures referred to by citation key, and the citations read out of the body rather than supplied. A reply containing any digit is a failed job, which makes "the model never produces a figure" structural in the one place it is hardest. Step 2 of `docs/plans/thread.md`
- **The conversation record.** `Tally.Threads` opens one thread per obligation, anchored so there is no blank box, and holds append-only messages with their author, body, transport and citations. A reply is unique on the message it answers, so a retried job writes nothing. Step 1 of `docs/plans/thread.md`
- **The advisor's briefing.** `Tally.Advisor.Briefing` ranks an entity's unsettled obligations into a list capped at three, states how many it held back, and returns nothing when nothing is due. Step 1 of `docs/plans/standing-advisor.md`, which is now a thin first cut: one channel (ntfy), a short ladder, and the delivery record, with email, the calendar, the call and the per-day controls left as additions. `docs/plans/thread.md` is cut to the same shape: text only, one anchor, no retrieval. `AGENTS.md` gains the principle behind the cut
- **The ntfy channel.** `Tally.Advisor.Ntfy` turns a briefing item into one push: the obligation, the duration and one action, with no amount and a priority that rises only with the real deadline. It goes through a new `Tally.Integrations.Client.deliver/3`, a delivery capability with its own declared endpoint, because a reminder alters no system of record. Step 2 of `docs/plans/standing-advisor.md`
- **The reminder ladder.** `Tally.Advisor.Ladder` picks the highest unsent rung whose threshold has passed, so a day the worker did not run is caught up rather than lost. `Tally.Advisor.Delivery` records what was sent once per obligation and rung, with the figure frozen and a state, so a rung fires once and a failed send is visible. `Tally.Jobs.Workers.RemindDue` runs it daily at 08:00 London, weekly once overdue. Steps 3 and 4 of `docs/plans/standing-advisor.md`
- **The forecast's goals and position snapshots.** `Tally.Forecast` holds the dated goals the ladder is measured against (`forecast_goals`: runway months, its milestone, and the deposit) and the append-only position snapshots that let a goal's date be seen to move (`position_snapshots`). Both are effective-dated and have no destroy action. Step 0 of `docs/plans/health-and-forecast.md`
- **The forecast rate.** `Tally.Forecast.Rate` computes the net monthly rate and the monthly cover from an entity's bank rows, with a source-of-record precedence (statement, then FreeAgent, then export), current accounts only, internal transfers excluded from both sides, and whole calendar months so a quiet month counts as a zero. It refuses fewer than three complete months rather than guessing. Step 1 of `docs/plans/health-and-forecast.md`
- **The forecast ladder.** `Tally.Forecast.Projection` turns the free cash, the rate and the goals into the ordered ladder: net positive, the Self Assessment reserve, the runway at twelve months of cover with six as the milestone, and the deposit. Each rung carries a date or `:not_at_this_rate`, and the score's band is the state so the two cannot disagree. Step 2 of `docs/plans/health-and-forecast.md`
- **The forecast loader.** `Tally.Forecast.run/2` loads an entity's bank rows, computes the rate, reads the reconciliation and the goals, and returns the ladder; `snapshot/2` records the inputs so a goal's date can be seen to move. `Tally.Banking.transactions_for_entity/2` is the entity-scoped read it uses. Step 3 of `docs/plans/health-and-forecast.md`
- **The health meter renders the ladder.** `health_meter` now takes the forecast and shows the score (marked estimated, so it cannot render like a measurement), the state, the track, and the ordered ladder with each rung's owner and either its date with its duration, "not at this rate" or "not yet tracked". Every state is on `/design`. Step 4 of `docs/plans/health-and-forecast.md`
- **The dashboard leads with the ladder.** The Position page opens with the health meter, the ordered goals with their dates or the reason they have none, and a plain sentence when the forecast cannot be worked out yet. The block that said the forecast was missing is gone. Step 5 of `docs/plans/health-and-forecast.md`
- **The daily position snapshot.** `Tally.Jobs.Workers.SnapshotPositions` records the position once a day at 07:15, after the bank feeds and the ledger post, so a goal's date can be seen to move. The "moved N days closer" line follows once there is a snapshot to compare. Step 6 of `docs/plans/health-and-forecast.md`
- **The dated cash-flow timeline.** `Tally.Forecast.Timeline` walks the known obligations in date order, grows the bank by the recurring net between them, and reports the projected balance after each, so a lump like Corporation Tax is not smoothed away by the average. The Position page shows each payment and whether the bank reaches it, and says "shortfall" where it does not
- **A goal's date shows how far it has moved.** `Tally.Forecast` compares the current projection to the last snapshot before today, recomputing the prior dates from the inputs the snapshot stored, and the meter shows "Nd closer" or "Nd further" once there is a yesterday to compare. Closes `docs/plans/health-and-forecast.md`, which is now implemented
- **Bank rows carry their provenance.** `bank_transactions` gains `source` (`:statement`, `:export`, `:freeagent`, `:monzo`) and a `source_snapshot_id` reference to the archived payload, and the rows already held are backfilled from what each already carried. This is what stops a FreeAgent row being mistaken for something the bank printed. `bank_accounts` gains a partial unique index on the bank's own account number, so two sources that name the same account differently can be recognised as one. Schema half of `docs/plans/tide-from-freeagent.md`
- **One resolver for a bank account, by the bank's own number.** `Tally.Banking.resolve_account/3` matches a source's account by account number first and name second, filling a missing number with the existing `identify` action, and both the CSV import and the FreeAgent import resolve through it. Without it a second source that names the same account differently would create a second account and split its history in two, invisibly
- **A paginated reader for FreeAgent's bank transactions.** `Tally.FreeAgent.bank_transactions/2` follows pages until one comes back short, returns each row with the archived payload it was read from, and returns an error rather than a truncated list when a full page does not advance
- **Reading the live Tide feed from FreeAgent.** `Tally.FreeAgent.BankImport.run/2` reads every active FreeAgent bank account, resolves it to Tally's own account by the bank's number, and records new rows with their source and the archived payload they came from. It pre-reads held fingerprints so a daily run writes only what is new, and matches the CSV-era overlap by amount and a date within one day, consuming each held row once, so a genuine same-day pair survives and the drifted rows do not double
- **The Tide feed runs daily.** `Tally.Jobs.Workers.SyncTide` reads the active Tide accounts from FreeAgent at 06:35, on the `:sources` queue beside the Monzo sync. It is gated on a FreeAgent connection rather than a live token, so the first request refreshes, and it raises a notice only when an account could not be read
- **The connections screen states the Tide feed.** The FreeAgent card now shows what bank rows Tally holds, the date range they cover and how long ago the newest is, so a bank three months behind reads as three months behind. The Monzo-specific empty state is scoped to Monzo, so it can no longer tell a FreeAgent reader to approve something in an app
- **A bank feed that has stopped is now visible.** A `:feed_stale` notice is raised when a current account has not moved for longer than three weeks, or when Tally holds fewer rows than FreeAgent reports. The stopped case is stated as a question, because a quiet account and a stopped feed look identical, and the connections screen is the action on it rather than Dismiss alone. A run that read every account clears the notice it did not reproduce; a run that failed an account clears nothing
- **The free-cash panel names the date its bank figures rest on**, and the connections screen says how many held rows are listed rather than proved by a statement
Fixed
"VAT" and "PAYE" are capitalised, and the words come from the domain
- **The reminder showed "Vat payment".** The obligation labels moved to `Tally.Obligations.label/1`, used by the reminder, the obligation list and the dashboard, so one place owns the vocabulary and a new screen cannot invent its own spelling
The ntfy push arrived as a raw JSON object, not a notification
- **ntfy renders a JSON body as the message text**, so publishing `%{title, message, priority}` to `/<topic>` showed the phone the object rather than a notification. The title, priority and tags belong in headers. `Tally.Integrations.Client.deliver/3` now takes a plain body and its headers, and `Tally.Advisor.Ntfy` builds them, with a test that the body is a string and never a JSON object
An overdue obligation was hidden from the cash-flow timeline
- **The dated walk filtered to obligations due in the future**, so a payment already past its date, the VAT twelve days late, was dropped entirely, which understated every projected balance behind it. An overdue obligation is still owed, so it is now placed at today rather than in the past
- **The stale-image check only looked at `mix.lock`.** The base image is rebuilt when the config changes, and a config change recompiles the whole application, so the run for that commit pays the cold-compile cost exactly as a `mix.lock` change does. The image now bakes a hash of `config`, and CI compares it, so the quality budget is skipped in that window too, which is the failure the `mix.lock` check exists to prevent arriving by another door
The ledger was not posted after a bank read, so the dashboard's bank figure was stale
- **The bank feeds now post the ledger when they finish, and check the result.** Reading a bank row and deciding what it means are separate acts, and posting was a manual `Tally.Release.post_ledger()`. On 2026-09-19 the Tide feed was brought current and the ledger was not, so the Position page showed £20,043.98 in the bank against a real £3,188.74, and free cash £16,855 too high, in the reassuring direction. The bank syncs now enqueue `Tally.Jobs.Workers.PostLedger`, which posts and then compares each account's ledger balance with the balance the source states, raising a notice on a divergence. The ledger was corrected in production the same day: Tide Current £3,196.17, Tide Instant Saver £2.39, total £3,188.74
The Tide sync read the company number from the wrong level
- **FreeAgent wraps every response in a key named after the resource**, so `/v2/company` carries the registration number under `"company"`. The worker read the top level, found nothing, and failed every run with `:no_company_number` — caught by running it in production rather than trusting the deploy. The read is now `Tally.FreeAgent.company_number/1`, with a test for the wrapped shape
- **A `Date` struct sorted by term order put 31 July above 18 September.** The Tide sync's newest-held row was computed with `Enum.max` on `Date` structs, which compares struct fields rather than chronology, so an account moving to 18 September was reported as stale since 31 July. It now sorts with `Date`, with a test for that exact pair. Found by running the worker in production
- **The free-cash date read the ledger account's id, not the bank account's.** The reconciliation's `bank_latest_on` was computed from the ledger account ids, which have no bank transactions, so it came back nil and the panel showed no date. It now reads each ledger account's linked `bank_account_id`, with a test
The nightly Monzo sync silently stopped reading
- **A connection with a lapsed access token no longer reads as no connection.** `Monzo.connected?/0` asked `Credentials.access_token/2`, which returns `{:error, :expired}` the moment the stored expiry passes even though the refresh token would renew it. The daily 06:30 sync gated on that answer, so when Monzo's six-hour access token lapsed it took the "never connected" branch, resolved its own notice and returned `:ok` without reading anything or ever reaching the refresh. It had been a no-op since 2026-09-11: seven runs, each completing in about 15ms, and no notice. The gate now asks whether the credential exists (`Tally.Credentials.connection?/1`), so the first request refreshes the token as designed. Two tests pin the distinction: an expired token is `{:error, :expired}` from `access_token/2` and still `true` from `connection?/1`
- **The same gate on FreeAgent, corrected before a worker could lean on it.** `FreeAgent.connected?/0` asked `Credentials.access_token/2` too. FreeAgent tokens last a week rather than six hours, so the stop would have come more slowly, but it would have come. It now asks `Credentials.connection?(:freeagent)`, with a test that an expired token is still a connection
Two silent under-reserves the weekly drift audit found
- **A missing salary source no longer reads as a zero.** A Director salaries workbook that is absent or unreadable left every invoice reporting PAYE as priced, with no reserve and no flag, because an un-ingested workbook was indistinguishable from a year with no salary. The source being unknown now leaves PAYE unpriced and counted, which is the first law's inaction half
- **An invoice that charged VAT but records none is refused, not reserved at zero.** The caller defaulted a blank `VAT amount due` cell to zero even when gross exceeded net, so the engine's `:vat_basis_missing` refusal never fired on the live path. A blank is now zero only when the invoice charged no VAT; otherwise it is left missing and the invoice is reported
- **The marginal Corporation Tax band says so.** A profit in the marginal band is priced at the main rate, and the provision's assumption now names that and says it is an upper bound, rather than presenting the figure as a complete estimate
- **The append-only promise is now the schema, not the absence of a caller.** The ledger and allocation references to their parent declared `on_delete: :delete` while `Entity` has a `destroy` action, so destroying an entity would have cascaded the ledger and the allocations away. They are now `on_delete: :restrict`, and a test proves an entity with an allocation cannot be destroyed while a bare one can.
Added
Corporation Tax figures now sit on the financial year, not the tax year
- The seed had been placing every figure on the 6-April-to-5-April tax year, but Corporation Tax is set per financial year (1 April to 31 March). On a rate-change year (19% to 25% on 1 April 2023), an invoice dated 1-5 April would have resolved against the previous year's rate, under-reserving CT. The four CT figures are now marked `period_kind: :financial_year` in the seed source, and `Seeding.period_dates/2` places them on 1 April while everything else stays on 6 April. A migration corrects the four already-seeded rows in place rather than re-seeding, because the `create` action's upsert identity would have left orphaned 6-April rows that overlap and read as `:ambiguous`. Plan at `docs/plans/ct-financial-year.md`.
Allocation and the free-cash figure (slice 2)
- Plan at `docs/plans/allocation.md`, accepted 2026-09-10. Delivers `Provision` and `Allocation` (append-only records decomposing an inflow into provisions plus free cash), the free-cash figure as bank balance minus outstanding obligations, and the dashboard headline that replaces the "cannot tell you yet" wait
- The decomposition separates three kinds of number rather than treating them alike: **exact** (flat-rate VAT), **estimated** (Corporation Tax), **heuristic** (PAYE/NI/pension). Stated and pinned by test
- A measurement settled the VAT question first, and it changed the shape: the flat rate is a dated, recorded fact about the company, **never** a category lookup. The books show 16.5% (the limited-cost-trader rate), not the 14.5% "Computer and IT consultancy" category rate, and a blended 0.819 ratio across the whole history — so no HMRC flat-rate table was seeded. See `docs/decisions.md` 2026-09-10
- **The decomposition engine, `Tally.Allocation.Decomposition`**, built 2026-09-11. A pure kernel that turns one inflow into VAT (exact, from the document's own figure or a recorded flat rate), Corporation Tax (an estimate behind a visible margin assumption, a per-invoice share and not a year-to-date true-up), and PAYE and pension (heuristics dropped outright once a real `paye_payment` obligation exists). No database, clock or rules table: the caller resolves the facts and passes them in, which keeps every branch testable without Postgres. A missing VAT basis is refused rather than left out, because leaving it out under-reserves. Percentages compute in `Decimal` and round once, so a fraction of a penny cannot leak through a rate. See `docs/decisions.md` 2026-09-11
- **The reconciliation, `Tally.Allocation.Reconciliation`**, built 2026-09-11. Free cash computed two independent ways, bank balance minus outstanding obligations against the sum of allocations, with a difference reported as a finding rather than averaged away. `:no_allocations` is its own state, so an unstarted ledger does not report a discrepancy and train the reader to ignore the warning. The comparison is pure; the loading half brings the ledger, obligations and allocations together, and the per-entity listing loads each relation once rather than once per entity
- **The Position page leads with free cash**, per bank-holding entity and never summed, because a company's bank and a person's bank are not the same pot. Each panel shows the bank balance and what is owed, the exact / estimated / planned split where a decomposition exists, and a plain sentence where none does. The figure is tagged as an estimate when what it subtracts is one, an entity with a bank and no recorded commitments says so rather than showing its whole balance as free, and the owed panel is named as every entity so two differently-scoped figures are not compared. See `docs/decisions.md` 2026-09-11
- **The caller, `Tally.Allocation.FromInvoices`**, and the two recorded assumptions it needs, built 2026-09-11. `Tally.Vat.FlatRate` is a dated flat-rate determination per entity and `Tally.Allocation.Margin` a dated Corporation Tax margin, both read as the latest at or before an invoice. The caller resolves them, the seeded tax rates, the year's salary and any real `paye_payment` obligation, decomposes each archived invoice and records the allocation, idempotent on the invoice reference. It refuses to invent an assumption: an invoice whose year has no margin recorded still posts its exact VAT, and is counted as unpriced for CT rather than reserved at nothing. This is the caller the split has been waiting for; see `docs/plans/allocation-caller.md`
- **Two gaps remain, and the slice-2 plan stays `accepted`.** The Corporation Tax estimate is a per-invoice share at a stated margin, not a year-to-date true-up (the year's profit is not visible to the engine), and the per-inflow deficit record the plan resolved is not built. Making the split render in production needs the margin recorded and the caller run, both of which are a `bin/tally eval` and neither of which is code. See `docs/plans/allocation.md`
The spec-driven harness, grafted from project-template
- Plan documents with a lifecycle in `docs/plans/`: a slice or feature starts as a **proposal** a human reads before any code exists, and `bin/check-specs` (a new stage inside `mix quality`) fails the build when a live plan's `Covers:` globs resolve to nothing or its code has moved long past the plan's last edit. Bugfixes skip this; their discipline lives in the commit message
- **Nine read-only reviewer agents** in `.opencode/agents/`, dispatched by diff shape rather than all at once: architecture, data, security, testing, performance, ux, dx, plus a drift auditor and an adversarial verifier. Each carries Tally's own constraints (the colour rules, the budgets, integer pence, entity scoping, the Gateway's no-tools rule) rather than generic criteria, and none can edit
- **`/plan`, `/review`, `/ship` and `/drift` commands**, encoding the workflow the sessions had been running by hand. `/drift` is the "snag round" formalised, writing its report to `docs/drift-<date>.md` instead of leaving it in chat
- **Two custom Credo checks** enforcing the architecture rules that were prose until now: `Tally.Checks.NoRepoInWeb` and `Tally.Checks.NoDirectExternalCallsInWeb` (no `Tally.Gateway`, `Tally.Embeddings`, `Tally.Integrations.Client` or `Req` from web modules). Both probe-tested against a deliberately bad file before being trusted
- **`[enforced: ...]` tags on the seven architecture rules** in AGENTS.md, with `bin/check-specs` printing the ratio: 5 mechanically enforced, 2 doc-only. A doc-only rule is visible debt rather than a quiet lie
- **A patterns library** (`docs/patterns/`) whose reference implementations the gate verifies still exist, seeded with the behaviour-plus-facade shape (`Tally.Gateway`) and the guard-test style (`test/tally/integrations/client_test.exs`)
The web/domain wall, enforced at compile time with `boundary`
- **Four boundaries replace prose.** `Tally` (the whole domain) exports 42 modules, the public API in one place: the context facades, three new root facades for the unrooted namespaces (`Hmrc`, `Capture`, `Fastmail`), and the types web legitimately holds (Ash resources passed to forms, `Accounts.Scope`, `Tax.RuleNotFound`, `SeedingFailed`). `TallyWeb` touches nothing else of the domain. `Tally.Application` is the top-level boundary supervising both. `Tally.Mix` holds the twenty mix tasks, top-level so the tooling does not inherit the domain's doors
- **Every measured crossing repaired, none parked.** 40 web-to-sub-module edges across 28 files routed through facades; 19 mix-to-sub-module edges the same way; the three OAuth `redirect_uri/0` functions that reached into `TallyWeb.Endpoint` deleted, with the integration controller deriving the URI and passing it in. Zero `dirty_xrefs`. The presentation trio (`pounds`, `in_words`, `precision`) moved from `Tally.Obligations.Position` to `Tally.Money`, which was already the exported home of money formatting, and the move surfaced a near-duplicate formatter that had been invisible
- **External doors closed with teeth.** Phoenix and Plug stay out of the domain, with `Phoenix.PubSub` the one named exception (the process broker domain events broadcast on). Ecto stays out of web except `Ecto.Changeset`. Runtime `Mix` calls are confined to the task boundary by a global check. Test support modules are ignored top-level boundaries, the pattern boundary's own documentation prescribes
- **The intra-domain map is committed** (`docs/plans/boundary-crossings.txt`, 360 cross-context edges grouped by pair) so promoting a pillar to a sub-boundary later is a diff against a record rather than an archaeology exercise. The pattern entry is `docs/patterns/boundary.md`, and the AGENTS.md rule carries `[enforced: compile]`
Fixed
The spec gate now catches same-commit plan drift, and the drift audit that ran found real drift
- `bin/check-specs` now carries a **same-commit check**: a commit that changes code under a live plan's `Covers:` but not the plan itself fails for an `implemented` plan and warns for an `accepted` one, and it runs in the pre-commit hook and the Claude Code hook (a bugfix that does not change the spec bypasses with `--no-verify`). The epoch check stays a 14-day warning as the coarse backstop.
- The drift the audit found is fixed: the caller plan now says `record_outcome/2`, the CT rate in effect at the invoice date, the real report shape, and that **pension is a named gap** (the engine provisions it but the caller hardcodes zero, so it is reported rather than silently under-reserved). `docs/plans/boundary.md`'s `Covers:` was narrowed from `lib/tally/**/*.ex` to the boundary declaration files, because a plan governing every domain module makes the check noise. A new `effective-dated-record` pattern names the resolver shape that had silently duplicated four times.
The facade check caught a live violation on its first run
- `TallyWeb.DocumentLive.Show` built its own `Ecto.Query` for a document's passages, bypassing the context layer the "contexts as facades" rule requires. Moved to `Tally.Archive.content_passages/1`, which is now the reference shape for getting a query out of the web layer
Monzo, for the personal account the company's books cannot see
- Slice 8 is the household — council tax, energy, water, internet, subscriptions — and **all three bank accounts Tally holds belong to the company.** Recurring-payment detection over them would find the FreeAgent subscription and some accountancy direct debits, and nothing the slice exists for
- **Direct to Monzo, deliberately not through an Open Banking aggregator.** An aggregator would give unlimited history at the cost of a third party receiving the account data, which is inconsistent with everything else here: embeddings run locally, the gateway sends `data_collection: deny`, and blobs sit under their own digest on this machine. Monzo's documentation permits exactly this use — *"your own account or those of a small set of users you explicitly allow"*
- Registered as a **confidential client**, which is load-bearing rather than a checkbox: *"non- confidential clients are not issued refresh tokens"*, and without one the connection would need re-authorising every six hours
The five-minute window, which shapes the whole design
- Monzo: *"After a user has authenticated, your client can fetch all of their transactions, and after 5 minutes, it can only sync the last 90 days."* There is one chance at the history, it opens when the owner approves in the app, and missing it means asking them to authorise again
- So `Tally.Monzo.Backfill` **fetches everything before writing anything**. A slow import, or one that fails on row nine hundred, cannot burn minutes that cannot be got back. It reports how long the fetch took and says plainly when the window was missed, because a run that finished in four minutes fifty worked and should not be trusted to work again
- `/transactions` caps at **100 rows** and says nothing about there being more — the same trap FreeAgent set, where one read returned 233 of 457 explanations and looked complete. Here it is worse, because a truncated page inside the window is history that cannot be re-fetched. Monzo also documents **neither the order pages arrive in nor how to tell the end**, so the cursor is the latest `created` seen, which asks for something strictly newer whichever order a page came in; every id is remembered, so a page repeating what is held ends the read rather than looping
What it refuses to write
- **A declined card payment.** It has a `decline_reason` and moved nothing; importing one would put a payment in the ledger that never happened
- A zero-amount row, which is an authorisation hold rather than a movement
- A row whose timestamp will not parse — reported, rather than dated today, which would put a 2019 transaction in this month's balance
Smaller things that follow from the shape of Monzo's data
- Amounts arrive as **integer minor units**, so nothing parses a decimal and nothing rounds — the one place in this codebase where money arrives in the right shape
- A row is fingerprinted on Monzo's own transaction id. Metro needs a positional fingerprint because a statement row has no identity; this has one, and it survives a re-fetch that a position could not
- A row reads as the merchant's name where there is one: *"TFL TRAVEL CH"* is what the bank wrote and *"Transport for London"* is what happened
- `approved?/0` distinguishes "waiting for you to tap approve in the app" from "the connection is broken", which look identical from a failed request — Monzo grants no permissions to a token until the owner approves, so a successful callback is not yet a working connection
Fixed
Sixteen snags in the Monzo slice, ten of them found only by connecting it
- **`description` is the user id, not an account name.** Monzo returned two accounts — a closed `uk_prepaid` from 2017 and the live `uk_retail` — and both carried `"description": "user_00009…"`. `Account`'s identity is entity, provider and name, so naming by it **collapsed them into one row**, with the live account inheriting `closed_on` from the prepaid card. Named from the type in words plus the last four of the account number now
- **Asking for nothing does not ask for everything.** `/transactions` with no `since` returned 88 rows covering one month, for an account opened in 2017 — and a page under the hundred asked for reads as the end of the list, so the run reported success on 30 days of a nine-year history. A backfill starts from the day the account was opened, which `/accounts` states
- **And the check that would have caught it**: the oldest row fetched, against the day the account opened, is the only independent test of whether the history arrived. A gap is a `GAP:` line and a notice rather than silence
- **Nothing checked that the archive and the ledger agree.** Answering *"is the whole history actually there?"* about the first real import meant computing it by hand: 1,235 distinct transactions Monzo sent, 1,063 held, 172 refused — 124 declined and 48 zero-amount — reconciling exactly. `Tally.Monzo.Reconcile` does it now, and the nightly sync raises a notice on any row Monzo sent that Tally neither holds nor refused for a stated reason. This is the distinction `Banking` already draws in its own words: per-statement arithmetic catches a misread row, and **the chain catches a missing statement, which no amount of within-statement arithmetic can.** The per-run invariant was the statement; there was no chain
- It reads the archive rather than Monzo, so it costs no request, cannot be defeated by an expired token, and checks the importer rather than the connection. It reports an unreadable snapshot rather than raising, because it runs after the import has already succeeded — and it never puts a payee or an amount in the report, only a date and an id, because that report becomes a notice
- **Two constraints on reading history, not one — and the first reading of this was wrong.** Two backfills came back with a month and then ninety days of a nine-year account. Measured: `since=2017` returns `400 bad_request.invalid_time_range` with `maximum_time_range: "8760h0m0s"` — 365 days per request — and `since=120 days` returns `403 forbidden.verification_required`. The 400 masked the 403, because asking for 2017 in one request fails the range check before it can fail the permission check. So `Tally.Monzo.windowed/3` walks the history in 300-day windows to satisfy the first limit, **and still has to run inside the window after signing in** to satisfy the second
- A `before=2020` request that returned 2019 data six minutes after authenticating was briefly taken as proof the documented five-minute window does not bind. The identical request fails at fifteen minutes. **One success is not a control**, and `docs/decisions.md` records the mistake alongside the answer because it is the more useful half
- Asking for **exactly** ninety days returns 403 for every account, so the fallback asked for a boundary Monzo refuses and read nothing at all. It asks for eighty-five now
- **A refused status is no longer retried.** The 403 propagated as a job error and spent fifteen of a forty-five *polling* budget, each attempt re-running a multi-request read against a bank. A status is the provider having declined the request, so it cancels with a notice explaining how to reach further back
- **Reaching back too far would have imported nothing.** A backfill asks from the day the account
- **Recovering a history meant deleting the token first.** Monzo's five minutes start at
- **The nightly sync would have asked about a card closed in 2017, every morning.** A backfill wants a closed account — nine years of household spending is the point — but a bounded read does not, and if Monzo refuses `/transactions` for a closed account that is a problem on the report and so a notice every single day. A bounded read skips them
- `Banking.held/2` counted rows in Elixir with a note saying an aggregate was the fix if it stopped being cheap. It stopped being cheap the moment an account opened in 2017 was about to be imported: it runs on the page somebody lands on after connecting, so it counts in the database now authentication, so re-running an import means signing in again — and the only route to the Connect button was Disconnect, which deletes the credential. That left somebody with no connection at all if the sign-in then failed, to recover something that was merely incomplete. A **Reconnect** button sits beside Disconnect now, safe for every provider because `Credentials.connect/3` replaces the row rather than adding one
- **The screen asserted a cause it could not know.** "Connected, but nothing has been read yet" told somebody to approve the connection in the Monzo app — which was wrong within the hour, said to somebody who had already approved it and whose rows had been deleted. It names both possible causes now, because distinguishing them needs a request it does not make opened, which for a 2017 account is years outside the ninety days Monzo allows once the five minutes have passed — and Monzo does not document whether it clamps such a request or refuses it. A refusal failed the whole read. It falls back to ninety days once now, and the gap check says how much is missing, because some history plus a notice beats none
- The field was **`type`, not `account_type`** — a guess whose fixture asserted a field Monzo never sends, so it passed while production recorded nothing
- **Every merchant name was being discarded.** Monzo sends `merchant` as an id string unless `expand[]=merchant` is asked for, so *"CARD PAYMENT TO TESCO"* was stored instead of *"Tesco"* — the whole point of a household view that groups by who was paid. The test passed because the fixture handed it a map
- **Dates were UTC.** A payment at 00:30 in summer was filed under the previous day, the previous month at a boundary, and the previous **tax year** at 6 April. Every other importer reads the date a bank printed, already London local; this is the one source that reports an instant
- **Ordinals were renumbered by every sync**, and `for_account` sorts by `[date, ordinal]`, so the day a thirty-day window began mid-way through came back with its later payments first
- **Unchanged rows wrote an audit event each** — three hundred a night describing nothing. A held row is now rewritten only when Monzo describes it differently, which is real: Monzo enriches a payment after it appears
- A foreign-currency amount is **refused rather than stored as pence**, one account failing no longer discards another's transactions, and the backfill runs in a queue of its own because Monzo's five minutes run whether or not a worker is free
A personal payment could silence a warning about company tax
- Three queries read **every** bank transaction with no entity scoping. Correct until 2026-09-10, because every account Tally held was the company's — so importing a personal account changed what they meant without changing a line of their code
- `UnpaidCharges` grouped outgoings by amount alone, and a match **suppresses** the notice. A household makes hundreds of payments a year, so one coinciding with a £127.75 penalty is close to inevitable over six years — and it would have silenced a warning about money owed to HMRC that accrues daily interest until paid
- `Missing` matched an unnamed-account commitment against everything, so a personal payment marked an HMRC Time to Pay instalment as met
- Both are scoped to the owning entity now, and **neither excludes the other entity's money**: a director paying company tax personally is recorded fact, so such a payment is offered as a candidate labelled with whose account it came from, rather than silently settling the charge
- Every existing test passed throughout, because each suite used a single entity — which is why this survived. Each fix now has a test verified to fail without it
A 401 left a connection dead for six hours
- `OAuth.token/1` refreshes on the stored expiry, which misses a token the provider invalidated early: that returns 401 while Tally still believes it valid
- **The retry alone would have been worse than the gap.** HMRC rotates its refresh token "on every use" and Monzo's refresh "invalidates the previous token", so refreshing more often raises the odds of two at once — and the second then presents a token the first invalidated, which kills the credential. For Monzo, reconnecting is what spends the five-minute window
- So refreshes are serialised on a transaction-scoped advisory lock with a double-checked read, and all four providers' identical `token/1` now delegate to one place
A successful import told nobody anything
- The backfill raises a notice only when something needs attention, and nothing in the web layer showed a bank account or a transaction. So the one five-minute window was spent and the only thing on any screen was the word "Connected" — eight thousand transactions and zero looked identical
- The connections screen states what is held: transactions, accounts, and the dates they span. It **updates live**, because it read that on mount and somebody arriving from the OAuth callback mounts it seconds before the import finishes
- Holding nothing is the useful case: it is exactly what an unapproved Monzo looks like
Two latent breakages
- `:monzo` was missing from `Credential.provider`'s `one_of`, so **connecting would have failed at the moment of storing the token** — after authenticating, inside the window. Nothing caught it because no test stored a Monzo credential
- `owner/1` derived whose account this is from "the sole `:person` entity", every night. Correct today and wrong the moment a second person exists, which this project expects: a partner is in scope for the household. It reads the entity the accounts are already filed against now
- The `unique` option on the backfill job **did nothing**: Oban compares the whole `args` map, and `authenticated_at` is a fresh timestamp on every insert
- And there were two `@window_seconds 300`, in the module that reports the window and the worker that polls against it, having to agree with nothing enforcing it
Trade debtors claimed £210,615.00 the company had been paid years ago
- 128 invoices raised against 50 receipts. **Not a bug in the posting**: `Tally.Ledger.Posting` is a cash book — every entry it writes has a bank account on one side — so a client payment recorded in 2018 has no bank row to post against, because the Metro statements begin in January 2021. It was correctly left alone, and the balance was the visible consequence
- The Crunch archive settles it exactly: invoices £719,718.40, less credit notes £24,563.40, less client payments £695,155.00, is **£0.00 to the penny**. Every invoice of that era was paid, and 117 client payments say so — 50 met a bank row and 67 did not
- `Tally.Ledger.Settlements` posts the other side of those 67 to `3100 Balance brought forward`, which is not a fudge: the money was received into an account whose January 2021 opening balance the ledger *does* hold, so it is already inside that figure. Dated at each payment's own date rather than lumped, so a balance struck in 2019 is right too
- **It asks `Posting` which movements went unmatched** rather than working it out again. A second opinion about what was already posted would show up as a double-counted receipt
- Only client payments. An unmatched expense is a cost the bank cannot evidence, and posting it here would put spending into brought-forward as though it were money received
A negative debtor is its own finding, not a reconciliation failure
- A test conflated the two and taught the difference. FreeAgent explained four 2026 bank lines as trade debtor settlements — £50,160.00 — and the invoices they clear are in FreeAgent rather than the Crunch workbooks. The ledger agreeing with the archive that £50,160.00 is owed *backwards* is not the same as the figure being right: both are describing the same missing invoice
- So it is reported on its own line and cannot hide behind "reconciles: yes". Booking a matching invoice nobody has read would be inventing a sale
Added
Tally asks for the VAT account now, because two sources disagree by £20,324.72 about it
- `2200 VAT payable` says £9,133.48 is owed. FreeAgent's returns say £29,458.20 across eight unpaid periods. **Neither can be trusted**, and the reasons are different: the ledger carries VAT charges from 2019 whose payments predate its bank data, and no charges at all after the Crunch handover; FreeAgent marks returns unpaid that were demonstrably paid — it categorised a £7,940.01 bank line as VAT and still shows the returns outstanding
- Corporation Tax had exactly this shape of problem and the captured HMRC pages settled it to the penny. No HMRC API publishes the VAT position either, and `vat-api` awaits production credentials, so the browser is the route — a page the owner is already looking at, captured, never a login automated
- The match fragment is **unconfirmed against the live service** and says so. The failure mode is safe: a wrong fragment means the popup does not recognise the page, which is visible, rather than capturing the wrong one
Fixed
Corporation Tax read as a £74,007.65 overpayment, and now agrees with HMRC exactly
- Booked from HMRC's own captured statement: £104,140.91 of tax, interest and penalties across eight periods, against £74,007.65 of payments already in the ledger, with **£30,133.26 brought forward** as the settlement made before Tally's bank data begins. `2320` now stands at **£0.00**, which is what HMRC's account says is outstanding
- £102,603.18 of that is tax on `8000`; **£1,767.57 is interest and penalties on `7910`**, visible separately rather than folded into a tax figure
- Trial balance is £0.00 across 3,024 lines
Added
The Corporation Tax account, booked from HMRC's own statement of it
- `2320 Corporation tax payable` held **six debits and no credits at all** — ten years of payments against nothing owed, reading as a £74,007.65 overpayment. It surfaced while moving an £18,515.19 payment out of suspense: the reclassification was right, and made the number worse, because a payment debits a liability nothing had ever credited
- **The charge deliberately does not come from the obligation's total, and that was tried first.** Charges and payments do not pair one to one — £14,857.43 charged against £14,842.37 paid, £9,968.92 against £10,033.32 — and the differences are *interest*, which a single `amount_pence` cannot separate from tax. Booking totals would have turned "£74,007.65 overpaid" into "£28,667.77 owed" when HMRC says every one of those periods is settled and the truth is nil
- So it reads `Tally.Capture.Statement`, which parses HMRC's own per-period breakdown — tax, interest, penalties, what was paid, what was reallocated — and **refuses a page whose figures do not sum**. Every statement is self-reconciling before it is booked
- Tax lands on `8000 Corporation tax`; interest and penalties on `7910`, so interest never reads as tax and penalties stay visible
- **The settlement Tally never saw** is one balancing entry to `3100 Balance brought forward`. Two 2019 periods were paid before Tally has any bank data, and booking their charges without their payments would invent a liability HMRC does not think exists. That entry asserts the payments happened on the strength of HMRC's own *"Less paid"* line — because if that account is good enough to say what was charged, it is good enough to say what was paid
- **It checks itself.** After a run `2320` equals the negative of what HMRC says is outstanding, exactly, and `reconciles?` reports whether it does rather than assuming it. Where it does not, the text says nothing downstream of the figure should be trusted
- What it does not claim: a balance struck at a past date. The settlement is dated at the start of the ledger's history rather than at each real payment, because HMRC's page gives amounts and not dates. The current position is right and the 2019 position is slice 11's problem
Added
FreeAgent explains the bank lines the Crunch archive could not reach
- `9999 Suspense` held £65,178.74 across thirty entries, and **the reason is a handover, not a defect**: `Tally.Ledger.Posting` explains a bank line from the Crunch archive, and that export ends 14 January 2026 when the company moved to FreeAgent. Every line after it had nothing to be explained by
- `Tally.Ledger.Explanations` reads FreeAgent's `bank_transaction_explanations` — the same kind of evidence, from the system the company actually moved to — and reclassifies each suspense entry it can account for. Nothing here is a guess or a model's opinion
- **The mapping is the only judgement, so it is a table.** FreeAgent's nominal codes to Tally's chart, every one of them observed in the real books rather than taken from FreeAgent's full category list: a mapping for a category the company has never used is an untested guess sitting somewhere authoritative
- `761 Bank Transfers in Progress` is **deliberately refused**, with the reason stated in the code. `Posting` pairs a transfer from both bank statements agreeing, which is stronger evidence than either system's say-so — and a transfer Tally can see one side of should stay a visible unknown rather than move to a clearing account nobody reconciles
- Exact on amount, three days on date: `Posting`'s rule, whose reasoning applies unchanged. A near-match on amount would be a reconciliation quietly absorbing a difference
- `dry_run: true` reports what it would move and writes nothing. The mapping moves real money between real accounts, so it should be possible to read what it decided first
Three accounts the real books needed
- `7910 Penalties and interest`, **its own account rather than administrative expenses**. Penalties are the thing this project exists to surface, and a chart that buries £964.33 of them among the stationery hides its most important number
- `1190 Other debtors`, because FreeAgent keeps trade and other debtors apart and a £27,000 receipt in the real books is classified as the latter — collapsing them would silently re-classify somebody else's accounting decision
- `6300 Subscriptions`
Fixed
A FreeAgent read returned 233 of 457 explanations and looked complete
- FreeAgent caps a page at 100 and says nothing about there being more. A single request omitted more than half the data, and **a truncated list looks exactly like an account with fewer transactions** — the consequence being a bank line left in suspense that FreeAgent could explain
- Three categories were invisible until it was fixed, two of them `371 VAT Penalty` and `372 Corporation Tax Penalty`. The mapping table would have been built without them
Added
A ledger entry can be reversed, which `Entry.reverses_id` had promised since the day it was written
- `Entry.reverses_id` is documented as "the only way to undo anything here", and **nothing had ever created one**. `Tally.Ledger.Posting`'s moduledoc went further and said `reclassify/3` "is what moves one, by reversing and re-posting" — describing a function that did not exist
- `Tally.Ledger.reverse/2` posts an entry's mirror, naming what it corrects. Dated as the entry it reverses, not today: a reversal dated later leaves the balance wrong for the period between the two, and a balance as at a date is what this ledger exists to answer
- Idempotent, and it refuses to reverse a reversal — undoing an undo is a re-assertion of the original and should be posted as one rather than left as a chain nobody can read
- `Posting.reclassify/3` is the pair of steps. **It moves only the contra side**: the bank side is what the statement says and reconciles to the penny, so a reclassification able to alter it could break the one check this ledger has against reality. The bank line is found by being one of the entity's bank accounts rather than by its position in the entry
- Needed because `9999 Suspense` holds **£65,178.74 across thirty bank lines** nothing explained at the time — including an £18,515.19 Corporation Tax payment sitting in suspense while `2320 Corporation tax payable` claims the company has overpaid by £55,492.46
Fixed
Nine HMRC penalty notices had no retention rule at all
- The most valuable documents in the archive — £964.33 of VAT penalties the app had no record of until they were read — and Tally could not say how long to keep them. The accounting-records rule named the books and not the correspondence evidencing a liability
- Checked against what the page enumerates rather than assumed: *"You must keep accounting records that include … **debts the company owes or is owed**"*, which is what a penalty notice and a collection letter are. Six years from the end of the financial year, on the same citation as the rest of that rule
- A test now asserts every kind a rule claims is a kind a document can actually hold. A rule naming a kind nothing can be governs nothing, and would read as coverage
Added
A payslip has a retention rule, and the gap was never really unfetchable
- It was recorded as a gap because HMRC's employer record-keeping *guidance* is not on the GOV.UK Content API under any slug tried. That was true of the guidance and false of the requirement: **it is regulation 97(1) of the PAYE Regulations**, and legislation.gov.uk publishes it. Refusing to guess was right; concluding it could not be sourced was not
- Three years, read out of the regulation's own words — *"must keep and preserve for not less than three years after the end of the tax year to which they relate"*. 106 payslips in the archive showed "no rule seeded" and now show a date with a citation
- **A new anchor, and the anchor mattered as much as the period.** "After the end of the tax year" is 5 April — not the 31 January submission deadline nine months later, and not a company's financial year end. Using either anchor already in the file would have been wrong by up to nine months, in the direction that destroys a record early. There is a test asserting the two are not the same date
- The documents page said it "does not know how long a payslip must be kept" and that the guidance "is not published where Tally can fetch it". Both were true when written and neither is now — the test that caught the last round of stale copy caught this one too
Added
A document has a page of its own, and it shows what the document says
- 883 documents were ingested, classified, read, indexed and searchable, and **none of them were readable in the app**. `/documents/:id/download` served the file, so the only way to find out what a letter said was to download it and open it in something else — and a search result could quote a passage while the document it came from had no page at all. The archive's premise is a figure with its source attached, and a source you cannot look at is half of that
- `/documents/:id` shows the text **as the search index holds it**, passage by passage with the character range each came from. Not a fresh extraction: that could differ subtly from what search matched, and then a hit would quote words the page does not contain
- The retention answer names its rule, quotes the sentence it was read from, and links to the source. A kind with no rule says Tally does not know, which is *not* permission to destroy it
- A file that arrived twice shows both arrivals rather than hiding them. Two routes is better provenance than one, and the digest proves they are the same file
- An absence is stated as an absence: a scan with no text layer says so plainly rather than rendering as a document that says nothing, and the identity passage is left out because the document never said its own filename
- Search results and the documents list link here rather than straight at the file. The download is one click away
Fixed
A penalty could be written twice, and a penalty changes a total
- `Tally.Ledger.Entry` states the standard in its own comment: *"a duplicated ledger entry is a duplicated figure, and unlike a duplicated document it changes a total"*, which is why posting refuses rather than upserts. **The penalties table did not meet that standard.** Deduplication was an in-memory check in `Tally.Documents.Apply`, and a check-then-write is not a constraint — two write paths reach the table, an approved reading and `Tally.Hmrc.Ingest`, each doing its own lookup
- One charge per obligation, kind, amount and date is now a unique index. It **enforces an assumption the code already made**: the existing check already treated those four fields as identifying a charge, so a genuine second one was already being dropped, silently. Now it is refused loudly, and the refusal reaches a person — `Apply` reports it and the job raises a notice
- `supersedes_id` is in the key with nulls not distinct, which is what makes it correct: two charges both supersede nothing and must collide, while two identical reliefs against *different* charges are genuinely different rows. Without that, cancelling two identical charges would lose a credit
- Verified against production before writing it: 7 penalties, 7 distinct on the key
Fixed
Every night, 66 MB of attachments were downloaded to discover we already had them
- The digest answers "have we got this" exactly, and needs the bytes to do it. So each pass fetched all 290 mail attachments, hashed them, and reported every one as already held. Not a bug in the check — the wrong question asked first
- A bundle now records `source_ref` (the JMAP `blobId`) and the pass asks that **before** downloading. A skip requires the ref **and** a matching size: JMAP says a blobId must not be reused for different data, and this does not depend on that being true, because the cost of checking is a column in an index and the cost of being wrong is *a document silently never archived* — the one failure nothing would report
- The 290 already-archived attachments are named by the ordinary pass rather than a backfill task: the bytes are in hand at the point it recognises one, so it records the id then and the pass after skips the transfer
- `Tally.Fastmail.Attachments` had no tests at all. It has some now, and they are entirely about the decision that skips work, including the same blob id at a different size
The same file, filed twice, was three answers in a search
- 77 files in the archive are filed two or three times: a bank statement arrives from the mailbox as `October 2025.pdf` and from the inbox ZIP as `2025-10.pdf`, byte for byte identical. Both rows are worth keeping and each carries its own provenance — **the arrival is a fact** — but they are one document
- Search and the documents list collapse on the digest rather than the row, so a search does not answer worse the more thoroughly something was archived. Nothing is deleted: the duplication cost no model spend, because the coverage register had already avoided re-reading the same bytes
Added
A bundle records the message it arrived on
- `source_message_ref` holds the JMAP `Email` id — provenance, and deliberately not the sender or the subject. An opaque id says *which message* without saying who wrote it or what they said, and that distinction is what makes it worth storing: this archive holds documents rather than conversations. Tally still cannot alter a mailbox, `Email/set` not being a permitted method
Fixed
Tally said board minutes could be destroyed after six years. The Act says ten
- Board minutes were governed only by the six-year accounting-records rule, taken from HMRC guidance because that was the only source Tally could fetch. **Companies Act 2006 s.248(2) requires them to be kept "for at least ten years from the date of the meeting"** — a longer period *and* a different anchor — and s.248(4) makes failure an offence by every officer of the company. The archive holds 83 sets, and the 2018 ones were already showing "keep until 2024-12-31"
- `legislation.gov.uk` is now a source alongside GOV.UK, for exactly this: **where guidance summarises a statute and the two differ, the statute is what a tribunal reads**. The period is read out of the Act's own sentence, not written from a constant
- `:board_minutes` stays on the accounting-records rule too. Two rules governing one kind is the case `Tally.Retention.for_document/2` was written for, and the longer winning is the point of it
- The weekly source check follows the statute as well, so a consolidation that changes s.248 is reported the same way a moved tax rate is
Added
`Tally.Sources` can archive a source that is not JSON
- `fetch_and_archive/3` takes `expect: :json | :text`. JSON stays the default and stays strict: a GOV.UK content path answering with HTML means something is wrong upstream, and wrapping that as text would archive a payload nobody could extract a figure from — a silent nothing rather than an error
- A text payload is wrapped under `"text"` rather than archived raw, so **every payload in the table still reads back the same way**. One source storing bare XML would break `payload/1` for every reader
- Statutes spell numbers out — s.248 says "at least ten years" — so the seed reads a word where a number is expected, from a short list. A word it does not recognise is a loud failure rather than something resolved to a plausible period
- A snapshot's published date now comes from `dct:valid` for legislation as well as `public_updated_at` for GOV.UK. A rule whose source cannot be dated is a rule nobody can check was current when it was read, and the seed still raises where neither is present
Fixed
561 of 768 documents held extracted text that search could not see
- The roadmap said correspondence was searchable. It was 27% searchable: 255 sales invoices, 134 bank statements, 83 sets of board minutes, 83 dividend vouchers and 5 credit notes had their text extracted by `pdftotext`, stored, and never indexed — 2.3 MB of it. Asking for the dividend declared in June 2018 returned the filename and never the resolution
- The cause: `Tally.Archive.Passages` asked `Tally.Archive.Readable.for/2`, which consults `Tally.Archive.Coverage` — whose job is deciding **whether to spend money on a model**. A `:duplicated` verdict on a dividend voucher means *"the figures are already in the workbook, do not pay to read this"*. It is right about paying and says nothing about text, and it was silently deciding whether free text reached the index
- **The same shape of fault as three others already recorded here: a cheap filter running before the thing that decides whether work happens.** So the two questions are now two functions. `Readable.text_only/2` answers "what text can this give for nothing", consults no coverage verdict, never renders a page and never reaches a model
- The owner's `local_only` veto still refuses. Indexing for local search is a different posture from never reading a document, but the veto is theirs to interpret and refusing is the conservative reading
Added
The review queue is sorted into decisions and acknowledgements, and the second half can be accepted a kind at a time
- Of the 175 readings waiting, **11 could write a charge** against an obligation and 3 more were demands for money. The other 161 were 106 payslips, 26 sets of board minutes, 5 dividend vouchers and a household bill — records to keep, none of which instruct Tally to do anything. A queue where the fourteen that matter are indistinguishable from the rest is a queue nobody starts, and one somebody skims is worse than one they never opened
- `Tally.Documents.Review` decides which is which, on two tests: **it would write something**, or **it came from somebody with a claim on you** — HMRC, Companies House, a debt collector. A document Tally cannot place counts as wanting a person too: `:other` means the filename told it nothing, and putting the unknown in the pile nobody reads is the wrong direction
- *"Carries a deadline"* was tried as a third test and **measured, then rejected**: 156 of the 161 remaining readings carry a date, because a payslip states its pay date. A signal that fires on 97% of what it sorts is not a signal
- "Would write something" is answered by `Tally.Documents.Apply.effect/2` — the same code that does the writing, never a second opinion. A bulk action deciding for itself that something is inert is precisely the failure to avoid, and the handler re-checks every reading at the moment it approves it rather than trusting the list the page was rendered from
- **Disputed readings are never swept up.** Two models disagreeing is the only signal Tally has that a reading may be wrong, and each group says how many it is leaving behind and why
- Each acceptance is a separate approval with its own audit event. Three documents were approved and the log should say three, not that somebody pressed a button
Fixed
175 readings were waiting for a decision and nothing said so
- The approve/reject queue lives on `/documents`, and the only way to learn it existed was to open a page named after storage and scroll past an upload form. The owner of this system asked whether that was even the right screen, which is the answer
- The header now carries a second badge — `n to judge`, linking to the queue — beside the notices one. **Separate, not added together**: a notice is something that went wrong and a reading waiting is ordinary work, so it is neutral rather than amber, and two numbers meaning different things must not become one number that means neither
- The page leads with what is waiting instead of with the upload form, and the queue panel now comes first. When nothing is waiting it says so plainly rather than silently omitting the panel
- Judging a reading refreshes the badge in the same breath, for the reason the notices page already did it: the header must not contradict the page underneath it
- `Tally.Documents.proposed_count/0` uses the `:pending` read action, which existed and described itself as "the review queue" and which nothing had called
Changed
The documents page put 175 judgements and 920 rows in front of you at once
- Both lists rendered in full. A queue of 175 readings to judge is a wall nobody starts on, and a table of every document below it is a page that scrolls for a minute and answers nothing
- Ten readings now, with the count of what is waiting and a button that says how many more it will add. The batch is defensible because of the existing sort: disputed readings come first, so the ten shown are the ten most worth a person's time rather than the ten inserted first
- Twenty-five documents, most recent first, and a pointer to search — which exists now, and is the right way to find a particular document rather than scrolling for it
- Neither count is announced when there is nothing hidden: a page saying "showing 10 of 10" is noise, and noise is what teaches somebody to stop reading the small print on a screen where the small print is the reason to trust it
Fixed
Two invoices were filed as arriving in the years 6529 and 6934
- `Tally.Archive.Ingest` reads a document's date out of its filename, and the pattern had no boundary — so in `invoice-676529-09-09-2025-15-28-20.pdf` it matched *inside the invoice number* and read the year as 6529. Nothing objected, because a `Date` is a `Date`, and the retention rule then offered to keep the document until the year 6535
- The pattern now requires a digit boundary on both sides, and a parsed date outside 1990 to next year is refused whether or not it parses. A parser producing a valid date has not thereby produced a plausible one
- `Tally.Documents.Repair` corrects the rows already written, to the date of the bundle each arrived in — the same fallback the fixed parser uses, and at worst a few days out. Run with `bin/tally eval 'Tally.Release.repair_received_on()'`
- A document that arrived in no bundle is **reported and left alone**. There is nothing to correct it to, and today's date would be a guess dressed as a fact — a visibly impossible date is better than a plausible invented one, because only the first one gets looked at
- The correction goes through `reclassify_document/3` and carries a note saying what happened, so the audit log records why a retention date moved by four thousand years
A document's date could not be corrected at all
- `reclassify` accepted the kind, the entity and the obligation but not `received_on`, so a document filed with a wrong date stayed wrong. Its own comment already made the argument — correcting *what* a document is and *whose* it is are the same kind of correction — and *when* it arrived is the same again, with more riding on it: the retention date is computed from it
Changed
A search result is now an account of a document rather than a window onto one passage
- The first version showed the single best-matching passage, which proved search worked and was not enough to use. A person scanning results is deciding **which document to open**, and that needs what a lone snippet cannot give: how much of the document answered, what the document actually is, and a way to open it
- A result now names the kind, who it concerns and when it arrived — `Receipt-2952-4119.pdf` says none of that — links to the file, quotes the two best-matching passages with the region each came from, and counts the rest: *"and 1 further passage of this document matched"*. One hit in a page of boilerplate and four hits through the body of a letter are different degrees of confidence
- The engines named on a result are now the union across the document. A letter whose body matched the exact words and whose covering page matched by a misspelling was found by both, and naming only the best passage's engine understated why it is there
Narrowing by kind, by who it concerns, and by the year it arrived
- Applied **inside each engine's query**, not to the fused list. Each engine returns its best thirty, so filtering afterwards would take a top thirty of everything and discard most of it — asking for one HMRC letter among forty payslips would find nothing, because the letter never made the top thirty. There is a test that fails on exactly that
- Only values the index actually holds are offered, each with a count of documents. A filter that can produce "nothing matched" from a valid choice reads as a broken search
- Filters live in the URL alongside the query, so a search is shareable and the back button undoes a narrowing. A stale bookmark naming a kind that no longer exists narrows to nothing rather than raising
Fixed
The same search returned its results in a different order each time
- Passages score identically far more often than the arithmetic suggests: every page of a bill that matches one word gets the same `ts_rank`. With nothing to break the tie, Postgres returned them in whatever order it liked — and because reciprocal rank fusion scores from *rank*, the fused order changed too. The best snippet shown for a query was effectively arbitrary between runs
- Every engine now breaks its ties by the passage's position in the document, then by id. Ordinal first because among passages that match equally well the earlier one is the better snippet, being nearer the subject of the letter
Search joined the documents table on every query to support filters nobody had set
- Measured: 2.2 ms of a 7.8 ms trigram query went on a sequential scan over all 920 documents to build a hash for a filter that was not applied — three times per search. The join is now made only when something narrows
Fixed
Two jobs that could not tell you they had stopped working
- `ReadUnread` and `BuildPassages` discarded their reports. An exhausted model balance produced 23 errored documents in one afternoon and a successful job; an embedding server that stays down means search quietly stops ranking by meaning. Both now raise a notice, which is the alerting channel — the mailbox `cron` would have mailed is not read
- `Tally.Documents.Extractor.needs_attention?/1` counts an errored document always, and a whole queue failing with nothing read — one document a model cannot make sense of is an answer about that document; every document failing is an answer about the route. A refusal never counts, since most runs are made of them
- `Tally.Archive.Passages.needs_attention?/1` counts passages left unembedded. A refused passage is an answer, not outstanding work, and a nightly notice repeating it would train somebody to skip the screen
Trigram search never used the index built for it
- `word_similarity(query, text) > 0.25` cannot use a GIN trigram index; only the `<%` operator can. Measured on the real corpus: searching `8779`, the kind of half-remembered reference this engine exists for, was a 128 ms sequential scan over every passage against **5.5 ms** with the index, returning the same eighteen rows
- The operator's threshold is a session setting, so it is set transaction-locally. A pooled connection outlives a request and a threshold left behind would change the next search
- A query that is an ordinary misspelled word still plans as a sequential scan, because at that point most of the table genuinely matches — and those are the queries full text answers well
- The length guard now counts characters rather than bytes: a query with a `£` in it is shorter than its encoding
The owner's veto on a document leaving this machine had no control
- `local_only` and `set_local_only` existed with no caller. `docs/decisions.md` grants the privacy exception that makes reading a scan possible — an image cannot be meaningfully redacted before it is understood — *alongside* that veto, so an unreachable veto is half of an agreement. The documents table now carries it per row, saying in both directions what the choice costs
- Marking a document local-only now deletes its indexed content passages **immediately**. The nightly build would have dropped them anyway, because `Readable.for/2` refuses a local-only document — but up to twenty-four hours is not an answer to somebody saying "hold this closely". The identity passage stays: it is the filename and the kind, not the contents, and removing it would hide a document from the person who filed it in order to protect it from them
A test suite that spent 28 of its 31 seconds asleep
- The embedding client retries transient failures with exponential backoff, which is right in production: the server answers 503 for about thirty seconds after a restart while it loads its model. Four tests that assert a failure paid for it. Read at compile time, deliberately — those tests replace the module's runtime config to point Req at a stub
Fixed
A retention period is now re-checked weekly, like every other sourced figure
- The three GOV.UK retention rules were seeded from live pages and then never looked at again: they were absent from `Tally.Sources.Check.sources/0`, so a changed period would have gone unnoticed indefinitely. A tax rate moving under a calculation and a retention period moving under a "may be destroyed after" date are the same failure, and the second is the irreversible one
- The check reads the period back with `Tally.Retention.Seeding.period/2` — the seed's own extraction — so the two cannot disagree about what a page says. A check reporting health while the seed refuses to run would be the worst of both
- Three findings are reported separately and none are absorbed: the number moved, the section is gone, or the sentence stating the period is gone. A rewritten paragraph must not read as "still 6 years"
- `Tally.Sources.Check` had no tests at all, which is an odd gap for the one thing that runs unattended — a scheduled job that compares nothing looks exactly like one that found nothing wrong
Two implementations of the retention date, and the tested one was not on screen
- The documents page had its own batched copy of the calculation because `for_document/2` runs a query per document. Two ways to compute a retention date is two ways for it to be wrong, and only one of them had tests. `Tally.Retention.for_documents/2` is now the single implementation, and a test asserts it agrees with `for_document/2` document for document
- A document no rule governs is absent from the map rather than present as nil — the same answer `for_document/2` gives
A comment that described plumbing nobody had built
- `Tally.Fastmail.Relevance` said a caller passed `statutory_years:` in from `Tally.Retention`. No caller did, and the option existed only to make the claim look true. Retention rules are keyed by document kind and a message is scored from a sender and a subject, so there is no honest mapping between the two yet — the comment now says that, and the unused option is gone
- `Tally.Retention.longest_years/1` went with it. A "longest period across everything" number is the kind of plausible-looking default the domain exists to refuse
Added
A reading now cites the line it was read from
- Slice 5 asks that extracted deadlines be shown against the region of the page they came from, and says why: *getting a date wrong from a letter is a first-law harm.* A reading saying "7 June 2026" is a claim; one that also shows the line of the document is evidence somebody can check in a second
- `Tally.Documents.Provenance` searches the passages, which already carried the character range each came from — no new storage, and the offsets were built for exactly this
- Matching tolerates the whitespace `-layout` puts inside a figure (`£ 18,889.35` against a transcribed `£18,889.35`) and **nothing else**. Not the comma, not the currency symbol: a figure that only matches after being rewritten is not the figure on the page, and a citation under a wrong number is worse than no citation
- A scan with no text layer says the figure could not be located, which is an absence rather than a reason to doubt the reading
Fixed
The documents page claimed three limitations it no longer had
- It said *"does not match them to obligations"* and *"does not know how long anything must be kept"*. Both were true when written and neither survived `Claim` and `Retention`, and the tests asserting them passed because the words were still on the page
- A page claiming a limitation it no longer has misleads as surely as one claiming a capability it lacks. The four bullets now state the limitations that are real: a reading never writes unattended, a figure cannot always be cited, a payslip has no retention rule, and an unmatched demand is not a settled one
Added
Six years of correspondence is searchable, by meaning and by string
- Local embeddings, on the GPU Immich's classifier already uses. This machine has AVX but **no AVX2 and no FMA**, which rules out much prebuilt ML tooling — and reasoning about instruction sets was the wrong approach, because Immich has been running ONNX Runtime with CUDA against the GTX 1050 Ti in the same compose stack for weeks. Tally uses llama.cpp instead of copying their runtime, for reasons about Tally: it dispatches CPU features at runtime and needs no Python, so the BEAM gains no native dependency
- `bge-small-en-v1.5`, 384 dimensions, 37 MB quantised, 92 MB of VRAM, in a container on the compose network with **no route out**. Embedding every passage of every document is a different privacy posture from sending the few pages nothing else can read, and no request header changes that
- **Three engines, because measuring found one insufficient.** Full text search for exact identifiers, vectors for meaning, trigram for a partly remembered reference or a misspelling. Fused by reciprocal rank, which needs rank order only — normalising a `ts_rank`, a cosine distance and a trigram similarity onto one scale would need weights, and weights need a benchmark to maintain
- One **identity passage** per document carrying its filename, kind, dates and the reading's issuer, reference and summary, so a filename or a charge reference is findable through the same index as the prose rather than through a code path per field
- `mix tally.passages`, `mix tally.search`, `mix tally.coverage`, `mix tally.spending`, and `Tally.Jobs.Workers.BuildPassages` on the daily chain after extraction
Every archived PDF can be read, where none could before
- 809 PDFs, 768 with their text extracted and stored, and **not one had ever been read**. Three components each behaved reasonably: `pdftotext` recorded a derivation, `Archive.unread/1` treated "has a derivation" as "has been read", and the gateway refused a PDF on the stated grounds that something parsed its text layer deterministically. Nothing did
- `Archive.PageImages` renders a scan at 150 dpi, which is the only route to the twelve PDFs with no text layer — **nine of them HMRC penalty notices**, found by filename months ago and never read. £964.33 of VAT penalties across seven notices, a Companies House statutory notice threatening prosecution, and £18,889.35 of Corporation Tax stated overdue
- `Archive.Coverage` is the register of what reads what, so a file nothing reads becomes a weekly notice instead of a silence. It reports `none: 0` over the whole archive
- `Crunch.ExpenseLines` reads the one workbook that says what any of the spending was *for*: accountancy is £9,128.80 of £16,420.07, 56% of everything
Changed
- `document_text` routes to `openai/gpt-oss-120b` with `z-ai/glm-5.3-flash` corroborating, on measured cost: 804 documents was about $4.00 against a $4.56 balance and is now about $0.40. Every candidate's price and accuracy is recorded in `docs/status.md`
- **GLM is no longer refused under the privacy floor.** It was, and every provider serving it collected prompts. Re-tested: `deny` now routes it via Wafer while `allow` goes to Z.AI
- A model is asked what a document says only where nothing else holds it. Kind-level coverage keeps 13 captured HMRC pages, 137 bank statements and 395 pieces of paperwork whose figures a workbook already states exactly away from a model — 785 documents would have been sent, and 222 were
Added
A search screen, so the index is reachable without a console
- `/search` over the whole archive. A charge reference, part of a filename, a figure, or a question in your own words. The query lives in the URL, so a search can be shared and survives a reload
- **Each result says which engines found it.** A result only the forgiving match liked is a guess at a misspelling; one the exact words found is a document that contains what was asked for. Both are worth returning and they must not look the same
- A content passage reports the character range it came from — *"characters 4,200–5,600"* — which is the provenance half of the roadmap's ask that a figure be shown against the region of the page it came from. An identity passage says it matched on the name instead, because it was assembled rather than extracted and has no position
- An unavailable embedding server is **stated on the page**. Passages are written before they are embedded, so the two string engines answer with no model at all, and the note says which are working rather than quietly returning worse results
- No red/green axis, per `docs/decisions.md`: the matched words are marked in cyan and every pill reads correctly with all colour removed
Added
`RetentionRule`: how long a record must be kept, with the page that says so
- Shaped like `TaxRule` because the discipline is the same. Three rules seeded from live GOV.UK pages, each carrying the page, its `public_updated_at`, when it was checked, and **the sentence it was read from**: company accounting records 6 years from the financial year end, VAT records 6 years, self-employed Self Assessment records 5 years after the 31 January deadline
- The seed **re-reads the sentence every run** and raises when the wording has gone, rather than writing a period whose source no longer supports it. It caught GOV.UK publishing a part titled `"Company and accounting records "` with a trailing space on the first attempt
- **PAYE is deliberately absent.** HMRC's employer record-keeping guidance is not reachable through the Content API under any slug tried, so a payslip has no rule and `for_document/2` says `{:error, :no_rule}`. Writing three years from memory is the mistake this domain exists to correct
- Every anchor lands on or after the date the record arrived, asserted for all twelve months: a document received in March belongs to a period that has not ended, and short is the direction that destroys a record during an enquiry
- `/documents` shows each document's date and the authority behind it, and says *"no rule seeded for this kind"* rather than a plausible date where there is none
- `mix tally.retention.seed` and `Tally.Release.seed_retention/1`
Added
A demand, set against what the books already say
- Slice 5 asks four things of a debt collection letter: what is claimed, by whom, whether it matches an obligation already known, and the real deadline against the stated one. The reading gave the first two and the extraction prompt already forbade conveying urgency; `Tally.Documents.Claim` adds the comparisons, **which are not facts about the letter at all** — a demand for money already paid reads identically to a demand for money genuinely owed
- Four verdicts. `:settled` is the one a person most needs before replying; `:known` means the demand is about a real liability; `:incomparable` means no amount and no reference were read. And `:unmatched`, which is **never presented as "this is not owed"** — nothing has been imported from every source, so the interface says so in as many words
- Matching is on exact pence or the same reference, however punctuated. Never on a nearby figure, because "about the right amount" is how a demand for one charge gets matched to another. Never on the issuer, because the agency chasing a VAT penalty is not HMRC
- Two records carrying the same figure are both returned rather than one being chosen
- **The letter's date is never corrected.** Where a matched record has a due date the two are shown together and their difference named: a collection agency may be stating its own payment deadline rather than a statutory one, and both are real dates about different things. What is refused is showing the letter's date as if it were the statutory one
- Shown only on a demand. A comparison box on a payslip says nothing and trains somebody to skip it
Fixed
A security test that sometimes tested nothing
- `tampered ciphertext is refused` overwrote one byte of the ciphertext with `<<0>>`, which is not a change when that byte is already zero. With a fresh IV per encryption that is roughly one run in 256: it failed once in several hundred runs and passed on every retry, which is the worst way for a security test to behave — it reads as a flake and is really a test that occasionally asserts nothing
- XOR rather than assignment, and an assertion that the ciphertext actually differs before checking that it is refused
Fixed
A rewritten passage kept the vector of text that no longer existed
- Improving how an identity passage is built rewrote 883 of them, and the run reported `embedded: 0, awaiting: 0`. That read as success while every one of those vectors described the old wording — search by meaning kept working and quietly answered about text that had been replaced. Nothing downstream could tell
- The upsert now clears `embedding`, `embedding_model` and any recorded refusal, **but only where the text actually changed**: `WHERE EXCLUDED.text IS DISTINCT FROM text`, so a daily run over unchanged text costs a query and re-embeds nothing
Fixed
A screenshot with no text came first for "what do I owe HMRC for corporation tax"
- An identity passage repeated the filename twice — once as `original_filename` and again as the archive path, which are the same string for most documents — and a third time in the compact line. A short passage that is its own name three times over scores well on trigram against a long passage that mentions the words once
- `:other` renders as "Something else", which said nothing about 89 documents and put the same two words into 89 identity passages. A generic label is noise wherever it appears
- The path is now included only where it differs from the filename, the label only where it means something, and the lines are deduplicated
Fixed
Searching for the whole of a filename found nothing, and a figure found the wrong document
- The generated `tsvector` normalised separators and the **query did not**, which is invisible until somebody types a whole string rather than part of one: `Receipt-2952-4119.pdf` indexed as four tokens against a query of one. The test written for it searched `receipt` and passed
- A comma splits a token where a hyphen does not, so `18,889.35` indexed as `18 889 35` against a query of `18889.35`. Measured: searching that figure returned a German warranty PDF and the document stating £18,889.35 of overdue Corporation Tax was nowhere in the top ten
- Both sides now call one `tally_searchable/1` function **in Postgres**, so there is a single definition of what normalisation means and no way to change one side without the other. A shared string interpolated into two Elixir fragments would have been the same bug waiting to recur
Fixed
- **A PDF was judged by the size of its wrapper.** The 40 KB floor was measured on *images* — logos at 1 KB against documents at 71 KB — and applied to PDFs it silently dropped 478 real ones, including a P60 and a draft letter to HMRC. A 2,663-byte PDF holding 873 characters is a complete invoice; compression is not evidence about content. Text is judged on density now
- **A limit bounded the list rather than the work.** `run_unread(limit: 50)` read nothing at all, because the queue began with fifty documents whose figures a workbook already held
- `Extraction.raw` was null on all 74 production rows despite being documented since the resource was written as keeping the model's own text for a reviewer. Nothing passed it
- `Archive.Chunk` tracked offsets alongside joined block strings and got both halves wrong — joining dropped the blank line between blocks, and it mixed `String.length` with `byte_size`, which diverge at the first `£`. Eight of twelve chunks from a real household bill failed to reconstruct. A chunk is a range now and its text is derived from it
- One over-long passage made the embedding server refuse its whole batch of sixteen: of 112 passages left unembedded, **98 were individually fine**. A refused batch retries one at a time
- `Readable` and `Archive.unread/1` forwarded the caller's whole option list into Ash, so `limit: 1` raised a Spark validation error from three frames deep
Fixed
A run that fetched nine accounting periods fetched 110 pages of HMRC
- `account/period/` is not a page, it is the prefix of a subtree: HMRC's legacy service puts `/period/7/tax`, `/interest`, `/payments`, `/repayment` and `/adjustments` under each period, each links to the others, and discovery ran on every page captured. Testing with `includes` admitted the whole tree and walked it
- Bounded three ways, because the want list alone was never a bound: a URL must match a want **as a page** rather than as a path prefix, discovery stops two hops from where the run was told to start, and a run stops at 40 pages however the matching behaves — saying so, rather than ending quietly
- Only the dropdown may name an accounting period. Discovery exists to reach a page that has one
- HMRC's own Welsh toggle was followed, after which the session language stuck and every later page was archived in Welsh with nothing saying so. A `lang` parameter is forced to `eng`, which refuses the switch and undoes it
- Ten periods counted as twenty-five, because `?lang=eng` and the bare URL were different keys. Identity ignores the query, on both sides
"May a run follow this?" is not "is this the want's page?"
- One function answered both, and a cold run captured two pages of twelve: HMRC's navigation links the bare landing page, whose path the want's match — with its trailing slash — never appears in
- Worse, the hub is a **different path**, not a looser form of the same one: the landing page is `/account/periods` (plural, no number) and a period is `/account/period/7` (singular). So a want names its `hub` explicitly. A hub is somewhere a run may go and read a dropdown; it is not one of the want's pages and does not count towards how many are held
- Reachability uses **every** want in the group, not only the outstanding ones. The route to the periods runs through the Corporation Tax statement, so satisfying that want removed the only link from the dashboard that leads anywhere — the chain worked solely in the state just after a reset
- Twenty assertions on both rules run in CI, covering the landing page, the subtree, the plural hub, query strings, fragments and the state pension prefix overlap
A sweep could not finish, then could not fail to
- `sweep` was added because one capture left the want reading "up to date" while nine periods were unread. It made the want *permanently* outstanding, so a run that fetched all ten still showed a hollow marker and "1 to do" — the opposite falsehood
- Tally could not know how many periods exist; the **page** knew all along. The reader now archives every dropdown as label/value pairs, so completeness is read from the captured evidence. Where a capture carries no dropdown the answer is nil and the want stays outstanding, because "cannot tell" and "complete" must not look the same
- Captures are ranked by `inserted_at`, not `received_on`. A run captures every page inside a minute, so ten same-day captures tied on date and the winner was whichever row came back first — which is how a later capture holding the answer lost to an earlier one that did not
Two surfaces, one job each
- Both were on screen at once offering the same errand with two different buttons. The popup is now the map — what is outstanding, what is captured, where each errand starts — and the in-page card holds every button and its own tick list
- The popup announced "This page — not on the list" for the Business Tax Account while the card below offered to send you to the page you were standing on. A starting page is on the list
- `extension/check.mjs` runs each script in CI. `node --check` only parses, so a stray top-level loop spliced into `popup.js` passed every check and still threw before defining a function, leaving the popup on its placeholder text with no error anywhere a person looks
Added
One command reads every captured statement, and a dry run is the way to start
- `mix tally.capture.read --dry-run` and `bin/tally eval 'Tally.Release.read_captures()'`, over `Tally.Capture.Statements.run/1` — the join that was missing while the reader and the ingest either side of it were written, tested and had no caller
- **Nothing is fetched.** Every page is already in the archive, put there by somebody looking at it, so this cannot be refused, rate-limited or asked about the wrong identifier — the reason it is a separate task from `mix tally.hmrc.sync`
- A dry run does the identical work in a transaction it rolls back. It runs the real code rather than describing it, which is the only kind of preview worth having: it is what found the two periods where an earlier ingest would have overwritten better information
- **It all reconciles, checked three ways.** Every one of the ten periods reconciles internally. The reallocations HMRC made between years sum to exactly £0.00, which independently confirms the sign handling. And the account as a whole — £102,603.18 of tax, £1,537.73 of interest, £104,141.88 paid, £0.97 of adjustments — sums to the £0.00 HMRC states
- **And it agrees with the bank.** For the one period where Tally's figure disagreed with HMRC's, two bank lines — £229.84 to Cumbernauld and £18,515.19 to Shipley — total exactly the £18,745.03 HMRC records as paid. The £72.24 gap between Tally's figure and HMRC's assessed tax plus that £229.84 is precisely the £302.08 of interest, so the conflict notice can be settled with certainty rather than judgement
Settled history is marked settled, and an amount conflict can be accepted
- A period HMRC has **not assessed** also reads `Total: 0.00`, and that zero says nothing about whether anything is owed — it says HMRC has not decided. Marking such a period paid claimed a £21,731.08 estimate had been settled. Only a period with an actual charge is treated as settled
- Reading nine years of Corporation Tax left seven obligations `:open` against deadlines from 2020 to 2025, so **£84,160.23 that HMRC's own page says is settled read as overdue**. The reasoning was right about the date and wrong about the debt: `Total: 0.00` is HMRC positively stating nothing is outstanding, and only *when* is missing
- `mark_paid_undated`, the mirror of `mark_filed_undated` — which exists for exactly this and had no payment counterpart. It records the state without fabricating the day, and the note is required, because "paid, date unknown" is a claim about a gap in the evidence
- `:amount_conflict` notices could only be kept or dismissed, never accepted: `/notices` reads `upstream_on` and refuses anything with no date. Notices now carry `upstream_amount_pence`, so a disagreement about money can be resolved in the source's favour with a person's click behind it — the same rule that governs accepting a date
Captured Corporation Tax statements are written into obligations and penalties
- `Tally.Capture.Ingest.corporation_tax/3`. `Tally.Obligations.CorporationTax` derives CT obligations from Companies House with **no amount**, saying "Corporation Tax owed is not knowable until the accounts are done" — true of a forecast, and untrue of history HMRC has been stating all along on a page no API exposes
- **HMRC does not simply win, and a dry run against the real account is what settled that.** A date HMRC states is the same fact as the date Tally derived, so `Tally.Hmrc.Ingest` overwrites it. An amount is not: the page states what was *assessed*, and Tally may hold what was *paid* — differing by the interest accrued on the day. Two real periods would have lost information: one where a figure corroborated by a payment confirmation *and* the bank line would have been replaced by an uncorroborated one, and one where a FreeAgent estimate would have been replaced by HMRC's silence
- So reconciliation only ever adds information: nothing held takes HMRC's figure, an estimate is superseded by an assessment, HMRC revising its own earlier capture supersedes it, HMRC's silence changes nothing, and **two sources that both have a claim write nothing** and raise the new `:amount_conflict` notice with both figures and the difference
- `:corporation_tax_unrecorded` is raised at last — declared since the resource was written and never fired by anything — and only where money is **still outstanding** for a period Tally held nothing for. Nine years of settled history is worth recording and not worth interrupting anybody about, so today it fires zero times and the day it matters it fires once
- Three things it refuses to assert: it does not invent a period start (the page states only the end), it does not mark a settled period paid (the page gives no payment date, and `paid_on` is "the day it was actually paid, not the day somebody told Tally"), and it does not record the live period as zero, which means *not yet* rather than *nothing owed*
- Repayment interest — a **negative** interest figure, which two periods have — is reported and not written. `Tally.Obligations.Penalty` allows a negative only as a `:relief` that supersedes the charge it relieves, and a captured page has no such charge to point at
A logo is not a document, and a model asked to read one will describe it
- 22 of the 47 readings waiting for review were *"Company logo"*, *"Award finalist logo"*, *"social media icon"*, *"Warning icon"* and *"Cartoon drawing of a truck"* — email signature images and page furniture archived alongside real correspondence. Each cost a model call and then a person's attention to reject
- Size separates them almost perfectly, measured rather than guessed: the junk had a **median of 1 KB**, the real documents a **minimum of 71 KB**. `Extractor.worth_reading?/1` refuses anything under 40 KB before any model is called
- The threshold errs low on purpose. At 60 KB it would catch one more logo; at 100 KB it starts blocking real letters. **A document silently never read is far worse than a logo somebody has to reject**, so two of the twenty-two still get through and no real document is at risk
- `force: true` bypasses it, for the case where something genuinely small needs reading
A reading can finally be judged, at `/documents`
- Proposed readings are shown with what the model reported **beside what it says it could not read**, and approving is what writes anything. Until now approval was reachable only from a console, so `Tally.Documents.Apply` could never run at all — the pipeline's third step existed and had no way to be reached
- **Disputed readings come first.** Where two models read the same page and disagreed, that is the one worth a person before any other: agreement is not proof, but disagreement is a definite reason to look. Eight of the current readings are flagged that way
- Rejecting requires saying what was wrong, which the resource has always enforced — so the button opens a field rather than rejecting outright. A rejection with no reason is indistinguishable from a misclick
- The page used to state plainly that Tally **does not read documents**, which stopped being true. It now says a model proposes and you decide, and that a PDF's text layer is read deterministically first — only a scan needs a model at all
The extension says when it read a page, rather than the server guessing
- A run paces itself at 1.2 seconds a page, so the moment a body reaches Tally is not the moment the balance was on screen. The extension now states `captured_at` and the server honours it, falling back to its own clock for an older extension
- It also removed a flake: two identical captures collapse onto one digest, that digest depends on the timestamp, and a double-click straddling a wall-clock tick produced two documents instead of one. The test asserting otherwise passed almost always, which is worse than failing
No provider that keeps what it is sent
- Every gateway request now carries `provider: {data_collection: "deny"}`. OpenRouter's default is `allow`, which permits endpoints that store prompts non-transiently and may train on them — and what goes through here is a council tax bill, a penalty notice, a bank statement
- **It refused something immediately, which is the point.** `z-ai/glm-5.3-flash` at $0.07/M read the test document well and is served only by providers that collect data; under `deny` it is refused outright rather than quietly used. `qwen/qwen3-vl-32b-instruct` at $0.10/M passes, served by Alibaba directly
- The models already configured are unaffected: Gemini 2.5 Pro routes to Google and Claude Sonnet to Claude Platform on AWS, both under `deny`. So this is a floor rather than a model choice, and it holds whichever model a route names
- Free models were investigated properly and are still not usable, now for a second reason on top of the privacy one already recorded: of eleven zero-priced vision models on OpenRouter, two are Google's *music* model, one is a content-safety classifier, and every candidate tested returned 429 or 403 or timed out
PDFs have a route at last, and it is free
- `Tally.Archive.TextLayer` and `Tally.Archive.TextLayers`, with `bin/tally eval 'Tally.Release.read_pdf_text()'`. **240 of the 281 unread documents were PDFs with no route at all**: the gateway is image-only by design, and the text-derivation machinery that existed could only ever serve the inbox archive, because it scanned a directory beside it
- **Measured before designed.** Twenty PDFs pulled at random from the archive: **twenty had a usable text layer**, at 195ms each under Ghostscript. These are born-digital documents, so sending them to a vision model would mean rendering a page to read text the file already contains exactly
- **`pdftotext` replaces Ghostscript**, and it is in the release image rather than on the host. Over the same twenty it was **9.3× faster** (387ms against 3,595ms) and never extracted less. Being in the image is what lets a PDF from *any* source be read — a mail attachment and a captured page had no route before
- **`-layout` is not optional.** Default output emits reading order, which destroys tables: on a real energy bill, *Electricity*, its dates and its £158.97 came back as three unconnected paragraphs with nothing saying which amount belonged to which line. With `-layout` the row survives, and because columns stay separated by runs of spaces it can be parsed deterministically as well as read by a model
- **Emptiness is measured by density, not size.** Two of the twenty produced 1,201 bytes containing 91 non-whitespace characters, all bullet points — a byte-count check passes that and records a derivation asserting the document says nothing. Both Ghostscript and poppler agree on those two, which makes it a property of the files rather than the tool: they are scans, and a vision model is the only route to them
The extraction pipeline has its missing third step
- `Tally.Documents.Apply` turns an **approved** reading into rows. `docs/decisions.md` describes the design as a model reading, a person approving, and deterministic Elixir applying — the first two were built, and the third did not exist. `mark_applied` was not even exposed on the domain, so every reading ever produced stopped where it was
- **A model cannot do this part.** A reading is deliberately generic — amounts and dates with free-text labels — because it has to describe a council tax bill and a penalty notice with one shape. Deciding that *"Late filing penalty: £100"* is a `:late_filing` charge against a particular year's return is a mapping between somebody's prose and Tally's schema, and that mapping has to be inspectable
- **Two gates before anything is written.** Approved, and only approved. And a matching obligation must already exist, because a penalty belongs to the thing it is a charge on — inventing one to hang a charge from would fabricate the debt it relates to. Where it is missing, a notice says so
- **The tax on a penalty notice is not a penalty.** A notice quotes what is owed as well as what was charged, and recording the first as the second would double the debt. Only labels naming a charge are placed; everything else is left alone
- With `OPENROUTER_API_KEY` finally set, the first document read out of the archive was an HMRC message stating a **Self Assessment late-filing penalty for the year ended 5 April 2025**, with £10 a day accruing from 1 May 2026 and the minimum already past £300
And how long to keep it, which is a different question
- `Tally.Fastmail.Relevance.retention/2` proposes a disposition for every message: `:permanent` for the terms a relationship runs on, `:statutory` for a record HMRC or Companies House can ask to see, `:useful` for evidence that money moved, `:disposable` for a notification of something recorded elsewhere, `:junk` for what never had a purpose here
- **Seven years, counted from the year after arrival.** HMRC asks six years of company records from the *end of the accounting period*, and the period a document arrives in has not ended — so the count starts late deliberately. Keeping a receipt too long costs storage; destroying one HMRC asks for during an enquiry costs a penalty
- Across the mailbox: 63 permanent, 442 statutory, 2,975 useful, 337 disposable, 3,192 junk — so about half could be cleared with a confirmation
- **A message from yourself is never junk**, which the first version got wrong: 878 messages from the owner's own domain — the largest single group in the mailbox — matched no rule, scored nothing and were proposed for deletion. The account's own addresses are asked of `Identity/get` rather than written into a rule table, because whose mailbox it is, is a fact about the account
- **Nothing can act on any of this.** `Email/set` is how mail is moved or deleted and it is not in the `read_over_post` list, so a retention verdict is a proposal for a person to confirm and the code physically cannot execute it. A test asserts that
Every message scored, with a sentence for every signal
- `Tally.Fastmail.Relevance` scores a message from its sender, its subject and its attachment names — all of which JMAP hands over **before anything is transferred**. Three bands: `:file` is worth archiving unattended, `:review` is worth a person's eye, `:ignore` is the rest. Across the real mailbox that is **563 / 1,002 / 5,444**
- Every signal contributes a number **and a sentence**, which is the part that lets a rule be corrected: a message in the wrong band can be traced to the rule that put it there
- **No sender reaches the file threshold alone**, and that is an asserted invariant. The first version scored the accountants one point over it and filed **all 448 of their messages** — every notification they have ever sent, when 40 carried a document. A sender says who is talking; it takes wording or an attachment to say this message is a record
- **Notification wording caps the band rather than subtracting from it.** 150 messages titled *"Statement Entries Uploaded"* were 21% of everything the scorer wanted to file, and a penalty as large as any positive still lost to two of them at once: 30 + 45 − 35 is exactly the threshold. A document that classifies overrides the cap, because an attachment is a record whatever the subject calls itself
- **Recurrence is the one signal a single message cannot show.** Eleven monthly payments from a processor are a subscription. Cadence alone was not enough — filtering for a monthly gap returned IONOS and Slack alongside MoneySavingExpert, Argos and Nectar — so a recurring charge must also carry a document, quote an amount, or come from a payment processor, and must not be sending through a marketing subdomain
- What it surfaces from nine years: 27 *VAT payment overdue*, 26 *VAT return overdue*, 20 energy statements, 14 accountant invoices, 14 hosting receipts, 13 *year end accounts are ready*, 9 Virgin Media bills, and *URGENT: Confirmation Statement is overdue*
A survey of all 7,009 messages, and what the classifier got wrong
- `mix tally.fastmail.attachments --survey` reports what every source would find and **downloads nothing**: JMAP hands over each attachment's name, type and size as metadata, and since documents are classified by filename, what is worth having can be decided for free
- **The mailbox, measured rather than guessed**: 7,009 messages over nine years, 1,099 with attachments, 1,431 non-inline attachments across 712 distinct names, 629 MB. Only **265 of the 1,431** look financial by name at all
- **There is no CT600 anywhere.** Zero matches in a filename, zero in a subject. The one independent statement of the Corporation Tax charge is not in email, and the filed accounts contain no tax figure either
- Sender filters had to be **tested against the live mailbox rather than reasoned about**. `from: "hmrc.gov.uk"` returns nothing — Fastmail matches by address and domain, not arbitrary substring — while `from: "hmrc"` finds twenty. And **41 of the 42 attachments in *Accounts* came from the accountant**, forwarding HMRC's notices on: an HMRC-only search would have missed every penalty notice found
- **`Tally.Archive.Manifest` filed all nine HMRC notices as `:other`**, and would have archived calendar invitations, GIFs and video as documents. Its rules were written for a directory somebody had already tidied; an email attachment keeps the sender's own filename. Now: HMRC's printed wording (*"Notice of first late payment penalty"*) is a `:hmrc_penalty_notice`, a statutory notice about an overdue confirmation statement is a `:companies_house_letter`, an engagement letter is `:accountant_correspondence`, energy statements are `:household_bill`, and `.ics`, `.gif`, `.mp4`, `.vcf` are excluded outright
- The phrase and not the word: a leaflet *about* penalties stays `:other`, which is what stops a guidance PDF being filed as a charge
The mailbox is readable, and it had seven HMRC penalty notices in it
- `Tally.Fastmail.Jmap` and `Tally.Fastmail.Attachments`, with `mix tally.fastmail.attachments` and `bin/tally eval 'Tally.Release.read_mail_attachments()'`. The OAuth handshake had held a working credential — mail scope, blob scope, offline access — for days, and **not one message had been read**: the door was built and there was no room behind it
- **What it found.** Four notices of a first late payment penalty, two of a second, a late submission penalty notice, a statutory notice about an overdue confirmation statement, and a letter about £18,889.35 overdue. Tally held **one** penalty notice in total. `Tally.Obligations` had computed £1,575 of exposure and said, correctly, *computed exposure, no evidence of a charge* — the evidence was in a mailbox nothing could read
- **JMAP has no GET**, so the read/write line cannot be drawn at the HTTP verb: one endpoint takes every request and the body says whether it reads. Blocking POST blocked reading Fastmail entirely; allowing it would have allowed `Email/set`, which moves and deletes mail, and Fastmail grants write because it offers no read-only scope
- So `Client.authorise/3` draws the line at the method names, against a `read_over_post` list in config. A caller declares what it is about to call and config decides whether those are reads — the existing rule intact, config the outer gate and no caller argument opening it. Declaring methods buys nothing from an integration whose config lists none
- Two things learned from the live session rather than from documentation: the API is on a **different host** from the one you authenticate against, and so is the download URL, so none is hardcoded. And a method-level JMAP failure arrives inside a **200** as an `error` response, which is how a mistyped filter looks like success until somebody reads the body
- No message bodies are fetched. What is wanted from a mailbox is the attachment; a body is prose
A deterministic reader for captured Corporation Tax statements
- `Tally.Capture.Statement.read/1` turns a captured accounting period page into figures in integer pence. **No model touches it**, and that is the point: the extension hands over a two-column table with fixed labels, already parsed, so sending it to a model would take data that is exactly right and make it probably right. A parser keyed on `"Repayments/Reallocations"` also *fails* when HMRC rewords the label, where a model quietly produces a plausible number
- It proves itself against three facts the page guarantees: the period stated in the prose must agree with what the dropdown says the period in the URL is; `Tax + Interest + Penalties` must equal `Sub total`; and `Sub total + Less paid + Repayments + Adjustments` must equal `Total`. A mismatch refuses and names the row, because if a label has moved every figure is suspect
- Verified against all eleven pages in the archive: eleven read, none refused. The reallocations chain correctly between periods — the £10,033.32 out of one year is the £10,033.32 into the next — and £1,537.73 of interest across ten periods becomes readable for the first time
- The reconciliation was wrong first time in an instructive way: it *subtracted* "Less paid", which the page writes already negative, so it passed on the one period whose "Less paid" was `0.00` and refused seven real pages out of eleven. Both settlement shapes are now tested
A captured account page is a statement, not a letter
- `:hmrc_account_statement`. Fifty-one captured Corporation Tax pages were filed as `:hmrc_letter` for want of anywhere better, which made them indistinguishable from actual post. A letter says a thing once; a statement supersedes the last reading of the same account
- Captures also archive the page's dropdowns and its same-origin links. The accounting-period select lists every period end date HMRC holds, which the page's table does not — and three times a run stopped because discovery matched nothing, with the links that would have explained it discarded
The extension's token is shown in Tally, and its endpoint is prefilled
- `/integrations` shows the capture token under *Browser extension*, masked until asked for — the same treatment amounts and tax identifiers get. Setup is now copy-from-app-to-extension, both in the browser
- **Why it matters more than it sounds:** Firefox discards a temporary add-on's storage along with the add-on, so the alternative was an SSH session and a `grep` **every browser restart**, to configure a browser
- The endpoint is prefilled in the extension, still editable for a development server. The token cannot be, because it is a secret and does not belong in a repository — that asymmetry is why one has a default and the other does not
- **Test connection** in the extension's options separates three failures that look identical from outside: the token refused, `TALLY_CAPTURE_TOKEN` unset on the server, and the endpoint unreachable. The last is almost always a wrong address rather than a wrong token
A guided list in the extension, driven by what Tally is actually missing
- **`GET /api/capture/wanted`** and `Tally.Capture.Wanted`. The obvious shape was a hardcoded checklist of pages; that would be a second copy of knowledge the database already has, and a tick against it would mean "you visited a page" rather than "Tally now knows this". It is computed from gaps, so **an empty list is the good state**
- **URLs are learned, not guessed.** Only `gov.uk/check-state-pension` is written down. The Business Tax Account's balance path is not recorded anywhere, and inventing one would produce a list item that never lights up with no way to tell why. A want carries a fragment; its URL comes from the newest capture that matched. The first capture teaches Tally where the page is
- **A state pension document satisfies its want**, so the list never asks for work already done another way
- **A tick claims only "captured and archived"**, which is known when the endpoint answers — not "read", because extraction runs later. The footer says so
- **Every exchange is logged.** Refusals always were; a success logged nothing, so the only evidence a capture worked was the extension's own badge, which is the half most likely to be wrong. The page is never logged, nor the token
- Scheduled `ingest_inbox` and `read_unread` at 06:45, after the Companies House refresh. Neither ran on a schedule, so three notices told the owner to drop a file in the inbox and promised the notice would close. **It did not**: nothing picked the file up
Tax identifiers on the entity, replacing an environment variable
- `national_insurance_number` and `self_assessment_utr` on `Tally.Entities.Entity`, which had promised these would arrive "with the access controls decided at the same time rather than retrofitted". `Tally.Hmrc.Sync` needs them, so the condition came due
- **`sensitive? true`**, as `Credentials.Credential` already does for a token: Ash omits the value from `inspect/1`, errors and logs, so a stack trace cannot leak one. Person-only, validated both ways. Normalised on the way in, so the spaced form HMRC prints is accepted. Masked on screen
- **`HMRC_NINO` is gone.** It was one identifier treated differently from its twin — the VRN has been a record with provenance all along — invisible on every screen, absent from backups and forgettable. Keeping it out of the *repository* is still right, but for git-history reasons rather than because it is personal
- **`/entities` and `/entities/:id/edit`.** An entity could be created and never seen again, so a value typed wrongly on the first day was unreachable. That is what made this blocking rather than untidy
The join between reading HMRC and writing Tally, which did not exist
- **`Tally.Hmrc.Sync` and `mix tally.hmrc.sync`.** Both halves of the pipe were built and tested and **nothing called the ingest** — so credentials would have bought nothing, and that was invisible until somebody grepped for callers
- Each source is independent. HMRC returns 403 for an API an application is not subscribed to, so a VAT failure must not stop the Self Assessment sync
- **A missing identifier is a skip, not a call.** The VRN comes from `Tally.Vat.Registration` and the NINO from `HMRC_NINO`, because personal data does not belong in the repository. A call quoting the wrong identifier is a question asked about somebody else
- A read that worked and an ingest that did not are reported differently: the first is HMRC's problem, the second is ours
`/commitments`, so expected payments can actually be recorded
- `Tally.Commitments` was built, tested and unreachable: the Time to Pay instalment could only be entered with `bin/tally eval`, which is the undocumented hand-made row this project already records once as a fault
- It shows the **schedule**, not only the failures, with a mark against dates nothing matched — so a wrong amount or day is visible before it becomes a notice. The mark is a sign as well as a colour
Notice text renders as Markdown, which it was always written as
- Every notice detail is written with `**bold**` for the sentence that matters, `` `code` `` for a field name, `##` for a section and lists for what to do — and it was rendered with `whitespace-pre-line` into a `<p>`, so **the asterisks and backticks were showing**. All that emphasis was not merely lost; it was noise
- `TallyWeb.Markdown` renders it, with a `.prose` stylesheet built from the design tokens rather than a typography plugin, for the reason daisyUI was dropped
- **The library choice was a security decision.** Notice detail interpolates bank transaction descriptions and HMRC API strings, so it is outside data reaching a page served from Tally's own origin. `earmark` was tried first and **emits a block-level `<script>` verbatim** whatever its `escape` option says — found by testing it, not by reading about it. `mdex` wraps comrak, where raw HTML is escaped unless `unsafe` is asked for, so the safe behaviour is the default rather than something a caller must remember
- Autolinking is off: a notice quoting a payment reference should quote it, not link it
A browser extension that captures a GOV.UK page in one click
- `extension/` holds a Manifest V3 add-on for **both Firefox and Chromium**: one `browser`/`chrome` shim, both background keys declared so each engine ignores the other's, and a `gecko.id` because without an add-on ID Firefox forgets `storage.local` on restart and the token with it
- **`POST /api/capture`** receives it, token-authed and **failing closed** when `TALLY_CAPTURE_TOKEN` is unset. A host allow-list restricts it to government tax services, so one token cannot make Tally general storage
- It archives through `Tally.Archive.Ingest.bytes/4`, the same door a file dropped in the inbox uses, so classification, extraction and ingest are all the pipeline that already exists
- **It sends structure, not HTML and not flat text.** HTML is refused by storage, correctly: a page served back from Tally's own origin runs in Tally's own origin. Flat text loses the pairing, since a summary list renders `Total amount due` and `£19,512.29` on separate lines. So it sends every `dl` and `table` as label/value pairs, which is generic Design System markup rather than knowledge of one page and does not go stale when HMRC rewords something
- **No content script and no field selectors.** Nothing runs while you browse; the page is read inside `scripting`, on your click. A selector for a specific number breaks silently on a redesign, and silence reads as "no balance owed"
`Tally.Commitments`, the one check that reports an absence
- Every other scan reads a record and reports what it says. This computes the payments that **should** have happened and looks for each one, which is the only way to catch a failure that produces no document
- A payment matches within a **tolerance** and a **window**, because interest moves a figure by pennies and a weekend direct debit lands on the Monday. One inside a six-day grace is not reported at all: the statement may not have arrived, and the notice people learn to ignore is worse than no notice
- **Each missed payment is its own notice**, unlike the state pension check where one question stays one notice. Here the count is the information: one instalment is a thing to fix, three in a row is an arrangement that has probably already collapsed
- The notice lists **what did leave the account** in the window, because the commonest reason a commitment does not match is that the amount changed
- Serves the personal side as much as the company's. A company's obligations have counterparties who write letters; a personal standing order has nobody chasing it until something has gone wrong
The HMRC integration reads, and writes what it reads into Tally's models
- **`Tally.Hmrc.Ingest`** turns HMRC's answers into obligations, penalties and notices — VAT and Self Assessment. Until it existed every wrapper was plumbing: HMRC could report a penalty Tally knew nothing about and Tally would not notice
- **It inverts the derivation.** HMRC's dates become the record and `Tally.Vat.Periods` the fallback for when HMRC is unreachable, which is safer because a derivation cannot know about a period HMRC changed
- **A disagreement survives.** Where HMRC's due date differs, the row takes HMRC's *and* a `:deadline_conflict` notice records what the derivation said. A difference on one period is a month-end rule; a difference on every period is a wrong anchor, and a silent overwrite would hide both
- **An accruing penalty is reported and never written.** Its figure is still growing, so recording it would put a number in the books that is wrong by tomorrow and dated as though final
- **The Time to Pay notice**, which is what the whole integration was for. HMRC returns the arrangement's window so Tally can say one is running without anybody remembering — and **not** the instalment schedule, which the notice says plainly rather than implying it is watched more closely than it is
- **`Tally.Obligations.StatePensionCheck`** watches the one deadline no API can reach. A gap in the National Insurance record is fillable for six tax years and then never, with no letter, no penalty and no overdue balance to announce it. Checked against HMRC's whole published catalogue: nothing returns the contribution record. A `:state_pension_forecast` document in the archive is the check
Every relevant HMRC API, wrapped read-only from its own OpenAPI document
- **Self Assessment Accounts 4.0** — balance and transactions, payments and allocations, penalties, charge history by transaction id *or* charge reference, coding out status
- **VAT (MTD) 1.0** — obligations, liabilities, payments, penalties, and penalty financial details by charge reference, which is the identifier printed on a notice
- **Business Details 2.0, Obligations 3.0, National Insurance 1.1, Marriage Allowance 2.0, Individual Benefits 2.0**
- **Write endpoints are absent rather than guarded.** A function that does not exist cannot be reached by a caller passing an approval
- **`Tally.Hmrc.TestUser` and `Tally.Hmrc.TestSupport`** create and seed sandbox fixtures through a separate `:hmrc_test` integration, so `:hmrc` itself stays read-only
- **`mix tally.hmrc.test_user`** and **`mix tally.hmrc.evidence`**. Running the sweep *is* the production application: HMRC grants access after seeing the calls in their own logs, and SDST must be contacted within two weeks. It refuses to run outside the sandbox, with no override
`BATCH_PROCESS_DIRECT` is the primary connection method
- Eleven headers, no browser, no public IP. HMRC's own description of the method is an overnight batch process fetching VAT obligations, which is what Tally is rather than what it resembles
- **`Tally.Hmrc.DeviceContext`** gathers the machine's half immediately before each call, which HMRC requires: a cached address under a fresh timestamp is a false statement, not a stale one
- **`Tally.Hmrc.ClientContext`** and a JavaScript collector remain for interactive calls, because a person pressing a button does initiate the action
- **The header validator needs no consent.** It is application-restricted, so `client_credentials` reaches it the moment the credentials exist — which makes validating the headers the *first* possible call rather than a later one
Fixed
- **`IBM Plex Mono` was declared and never loaded.** `.num` and `.prose code` named it, no `@font-face` existed and no file was vendored, so **every figure in a product made almost entirely of figures** silently fell back to the system monospace. Nothing broke and nothing warned. It is now self-hosted (two weights, latin subset, 30kB) as `docs/design.md` requires and the CSP enforces, with the OFL licence alongside it
- **`TallyWeb.FontsTest` makes that class of fault visible**, since a declaration resolving to nothing cannot be caught by a compiler. It asserts every quoted family has a face, every face's file exists and is really woff2, and no external font host appears
- **`StatePensionCheck` ignored the purpose-built model.** It looked only for a `:state_pension_forecast` document, so a typed `Tally.Pensions.StatePensionRecord` with no PDF beside it left the notice open — telling somebody to go and do what they had already done. It now takes the newest of a reading and a document
- **`StatePensionRecord`'s moduledoc promised a `stale?/2` that was never written**, which is worse than a gap: a reader would have believed the question was answered somewhere. Staleness lives in the scan, and the moduledoc now says so
- **`FraudPrevention.list/1` percent-encoded the separators inside every entry it joined**, so `Gov-Client-Screens` came out as `width%3D1920%26height%3D1080`: the right length, the right shape, and completely wrong. It survived because the function had no caller and its test only ever passed it plain words. `encoded_list/1` now handles the lists whose entries genuinely do encode
- **HMRC amounts decoded to floats.** `0.29` as a float is `28.999999999999996`, which truncates to 28 pence against HMRC's 29. `Client.decode/1` now decodes with `floats: :decimals`, so float arithmetic on a tax figure is impossible by construction
- **`Client.get/4` passed the caller's whole keyword list to the HTTP client**, so `:method`, `:actor` and the vendor keys leaked into a library that rejects unknown options
- **The client promised `{:missing_fraud_headers, names}` and never produced it**, passing the inner `{:missing, names}` through, so a caller could not tell a missing header from a missing token
- **Test config inherited the real base URLs**, so a test that forgot to stub the HTTP layer made a genuine request to HMRC's sandbox — which happened, and HMRC answered. Every integration now points at an unroutable `.invalid` host in tests
- **Four test modules mutated `config :tally, :integrations` while async**, wiping every concurrent test's configuration. Latent until tests existed that needed HMRC configured
The HMRC integration, starting with the part that gets applications rejected
- **`Tally.Hmrc.FraudPrevention`** builds the sixteen `Gov-Client-*` and `Gov-Vendor-*` headers HMRC requires of a `WEB_APP_VIA_SERVER` connection. Built first, not last: HMRC grants production access *after* being satisfied these are accurate, so discovering a problem here at the end of a six week process is the failure mode to avoid
- **It refuses to send an incomplete set**, naming what is missing, because a partial set is exactly what HMRC judges and failing a call in development beats failing an approval
- **`placeholder?/1` rejects `null`, `undefined` and friends**, which HMRC forbids explicitly. A JavaScript collector produces them by accident, and a placeholder is **worse than an absent header**: HMRC reads it as a real value
- **Deliberately empty is a third state.** `:none` in config means "cannot be collected, agreed with HMRC"; absent means somebody forgot. `Gov-Vendor-License-IDs` is the live case
- **`Tally.Hmrc.OAuth`** follows the FreeAgent pattern exactly. Every scope is a `read:` scope, so the credential itself cannot submit — the strongest form of the read-only rule, and stronger than FreeAgent where no read-only scope exists. HMRC **rotates the refresh token**, which FreeAgent does not, so the new one is stored or the connection dies at the next refresh
- **`Tally.Hmrc.Client`** is the only door, and it will not make a call it cannot supply fraud headers for. The API version is a **required argument**: a caller that has not chosen one has not finished thinking about the call
- **Sandbox by default**, production opted into by `HMRC_ENVIRONMENT=production`. An integration that silently reached the live tax account because a variable was unset is the wrong way to fail
Changed
Recommended against taking HMRC write access to gain a read
- Read-only is disqualifying for VAT (MTD), so the obvious move is to build submission and meet the standard. Recorded in `docs/decisions.md` as a recommendation against, with the reasoning, because it is the owner's call and it changes what the product is
- **The price is out of proportion**: the reason to want the API is to read liabilities and penalties, and acquiring that by taking on the ability to file is the highest price available for a small convenience
- **FreeAgent already files.** Two systems able to submit the same return is a genuine double-submission risk, and a return filed twice is not undone by reverting a deploy
- Building it and never enabling it would misrepresent the product to HMRC, which is a bad idea on its own terms
- **A second door may also be closing.** HMRC's Self Assessment Accounts page now warns that production credential requests for new 2026–27 quarterly update products are no longer accepted. The fallback plan rests on that API, so `docs/status.md` now says to email SDST **before** committing to any of it
Fixed
The header list was written from memory and was wrong twice
- Checked against HMRC's own page rather than trusted. **`Gov-Client-Browser-Do-Not-Track` does not belong to this connection method** — and because an incomplete set is refused, **every call would have been blocked** waiting for a value nothing was going to collect
- **`Gov-Client-Public-IP-Timestamp` was truncated to the second.** HMRC specifies milliseconds. The truncated form is valid ISO 8601 and does not match the specification
- Neither would have been caught by a test written from the same memory. **The tests now use HMRC's own literal examples**
Read-only is disqualifying for VAT (MTD), which was worth knowing before applying
- HMRC's VAT (MTD) guide requires software to both retrieve obligations **and submit a VAT return**. Both mandatory; retrieving penalties and liabilities are optional
- So the API whose liabilities endpoint would answer the `charge_unpaid` question **cannot be approved for a tool that does not file**. Recorded in `docs/decisions.md` with the three ways out, none of them free
A drop folder, so adding a document does not require an archive
- `/mnt/raid/tally-inbox/drop/`. Anything put there is ingested on the next run, sub-directories walked, so a folder of printed pages can be moved in whole
- **A loose file is its own bundle of one**, which keeps the invariant that a bundle's digest is the digest of what was handed over. A standing bundle that files are added to would need a synthetic digest, and a digest that is not of anything is worse than none
- It exists because the inbox knew only two ZIPs by name. **A screenshot of an HMRC account page put there would have been ignored in silence**, which is the failure this whole domain is built to prevent
"Was the charge paid?" — a question nothing was asking
- **`Tally.Obligations.UnpaidCharges`** could not exist before today, because no `Tally.Obligations.Penalty` row existed anywhere. The first one arrived from an HMRC notice and made the next question obvious: an unpaid charge **grows**, and HMRC's own notice says a second penalty accrues daily at 4% a year plus interest until paid in full
- New notice kind **`:charge_unpaid`**, resolved by a later scan finding the payment, like every other kind
- **It matches on the amount and says plainly what that does not prove.** Five ways a paid charge still shows here are listed in the moduledoc, and the first is the common one: HMRC routinely collects a penalty inside a larger payment
- So the notice lists **candidates** — every payment to HMRC on or after the charge date large enough to have contained it. **A short list beats a six-year statement**
- **It never computes what the charge has grown to.** `Tally.Interest` could, and must not: the figure would rest on a balance and a settlement date the two sources disagree about, which is a confident number built on a contradiction
- Honest about being a stopgap. The robust answer is HMRC's own liabilities and payments endpoints, which the sandbox application is already subscribed to and which are four to six weeks away
The first real penalty charge, and two documents this repository described wrongly
- **`Tally.Obligations.Penalty` has its first row.** Every notice has said *computed exposure, no evidence of a charge* since the day it was built. **£127.75**, an HMRC VAT late payment penalty charged 2025-01-09 for the quarter ending 2024-10-31, reference `XS006134524675`
- **Two things it raises are deliberately not recorded as figures**, because HMRC stated none: a **second** late payment penalty accruing from 2025-01-07 at 4% a year plus interest, whose amount HMRC said would be confirmed separately and for which **no notice exists in the archive**; and a flat contradiction — Crunch records that quarter paid 2024-12-30 and HMRC wrote on 2025-01-09 that full payment had not been received
- **`VAT late penatly point.jpg` is not a points notice** and says nothing about the 2025-10-31 quarter. `docs/status.md` said it corroborated that quarter being 64 days late. It does not
- **`payment-for-late-nics-debt-collection.png` is a receipt**, not an exposure: £471.38 paid successfully to Past Due Credit Solutions. `docs/status.md` called it "the only item in either archive that looks like an exposure nobody has mentioned"
- **Both descriptions came from filenames rather than from the documents**, which is the pattern rather than two separate mistakes
Five readings recorded as proposals, with honest provenance
- Read interactively rather than through `Tally.Gateway`, because no `OPENROUTER_API_KEY` is set, and recorded under provider `claude-code (interactive)` with `corroboration` **nil rather than empty**: one reading, and no second reader checked it
- A reading taken by hand is still a reading. **A penalty recorded from a document nobody can trace back to a reading is a figure with no provenance**, which is what the extraction log exists to prevent
- The council tax bill confirms the February trap the manifest records: **£2,391.59 in ten instalments, February and March free**
- The flat rate VAT return shows **box 4 reclaimed £0.00**, independently confirming the decision to post expenses gross under the scheme
The 20 historical VAT quarters are imported
- **82 obligations, up from 39.** 29 returns and 24 payments, back to 2019
- **37 open notices: 24 live, 13 historical.** The ranking is what made the import safe to do
- **Two of the live ones are large.** A VAT payment **393 days late** with its window open to 2027-01-31, and one **351 days late** with its window open to **2026-10-31**, under two months away
Fixed
Payment obligations were skipped for every quarter whose return was already held
- `unmatched` is return-centric, so a quarter Tally already had a return for was never considered for a payment obligation. That skipped the **nine most recent quarters** — the ones most likely to be inside HMRC's assessment window — and one of them carries the £127.75 penalty above
Notices can tell a live exposure from a historical one
- **`Position.assessment_deadline/1`** and **`assessable?/2`**: four years from the end of the period, which is the general limit under the Taxes Management Act and the VAT Act
- **It ranks a notice and never closes one.** The window extends to six years for careless behaviour and twenty for deliberate, and Tally cannot judge which applies, so four years is the earliest date an exposure *could* be time-barred rather than proof that it is
- **Nil means assessable**, which is the safe direction. A Companies House penalty is automatic on the filing date rather than assessed, and reading "no window computed" as "time-barred" would hide a live exposure. An absence of a limit is not an expiry
- A historical notice says so in its summary and explains in its detail which extension an assessment would now need
The 20 historical VAT quarters can be imported, behind a deliberate flag
- `TALLY_IMPORT_HISTORICAL=1` creates the quarters back to 2019 that Tally has never held: **11 filed late and 14 paid late**, none of them previously visible
- **Two obligations per quarter, and the second is the point.** A return carries the filing date and a payment carries the payment date, and they diverge badly: the quarter ending 2023-01-31 was **filed 17 days late and paid 393 days late**. Recording only the return would report the smaller of the two and call it the exposure
- **A payment obligation carries no amount.** `Tax payments.xls` records payments without saying which period they settled, and one day holds four separate VAT payments, so the lateness is known and the sum is not
- A quarter with no payment date gets no payment obligation, rather than an open one asserting money is still owed on a 2019 return
- It was gated on the window existing first. Twenty notices for time-barred periods sitting beside Monday's Time to Pay instalment is exactly what the ranking is for
`Tally.Ledger.Journals`: the ledger holds both sides now
- 477 entries from the archive — 128 sales invoices, 170 expenses, 83 dividends, 91 salaries, 5 credit notes — and the trial balance is still exactly £0.00
- Until now every entry had a bank account on one side, so `Trade debtors` held only the receipts and never the invoices that created the debt. That was written into the moduledoc rather than left to be discovered, and it is now closed
- **A sales invoice has three credit lines because of the flat rate scheme.** The company charges 20% and pays HMRC a flat percentage of gross: £1,200.00 charged against £1,188.00 due on the same invoice. **The £12 difference is income**, not a rounding error, and posting the charged figure would have overstated the VAT liability on every invoice for six years. It goes to `4100 Flat rate scheme surplus`, which comes to **£2,015.40** across the archive
- **An expense posts gross**, because flat rate input VAT is not reclaimable. Nothing accumulates in `VAT recoverable` that the company cannot claim
- **An expense payment from a bank row now debits `Trade creditors`, not the expense account**, so the invoice is the only thing that ever charges a cost
Fixed
An opening balance was posted over history the ledger already held
- Tide has one statement covering March and CSV rows from February, so posting the March opening balance of £12,540.00 counted February twice and the reconciliation was **out by exactly that**
- An opening balance is only valid where nothing precedes it. Metro never showed the bug because its earliest statement is also its earliest data, which is the shape that survives a long time: correct on the dataset it was written against, wrong the first time a second source arrives
Dividends presented as a negative £421,088
- `presented/2` flips equity so it reads positive, but a dividend **reduces** equity and normally carries a debit balance. `Account.contra` marks it, set by what the account is for rather than inferred from which way its balance happens to be sitting
Every document is read twice, by models from different families
- **The fallback chain protects the format, not the answer.** A misread figure in well-formed JSON passes schema validation and every other check in the system. The only control that catches it is a second, independent reading, and `docs/decisions.md` specified exactly that before any of this was built: where two readings disagree about a figure, that is a flag for human attention rather than a coin toss
- `Gateway.extract_document/2` takes `corroborate:`, **on by default**, and reads the page with the primary model and the next in the chain — deliberately a **different family**, because two models of one lineage fail the same way often enough that agreement between them means much less than it appears to
- **Compared on figures and dates, never on prose.** Two summaries always differ in wording, and comparing them would report a disagreement on every document. Amounts are compared as sets of pence rather than pairwise by label, because two models label the same figure differently
- **`Documents.disputed_extractions/0`** is the top of the review queue: proposals where the two readings differ about a number on a financial document
- **Nil and zero are different claims throughout.** Zero disagreements means two readings agreed; nil means there was no second reading and nothing was checked. A second reader that cannot answer leaves the first reading standing and records the absence of the check rather than an agreement
Changed
The stronger model goes first, which reverses the original routing
- `google/gemini-2.5-pro` primary, `anthropic/claude-sonnet-5` behind it. Sonnet 5 is both newer and cheaper than the 4.5 first configured, $2.00 per million against $3.00
- The first draft put the cheap model first on the reasoning that a fallback catches what it cannot handle. **That reasoning is wrong**, for the reason above, and the saving was fractions of a penny
- No dedicated OCR model is available on OpenRouter. `qwen/qwen3-vl-235b-a22b-instruct` at $0.21 per million is the only model in the catalogue naming document understanding, and is worth knowing as a tie-breaker a person consults. **Not as a third vote**: three models resolving by majority is the coin toss the decision rejects
Three bank statement formats, one algorithm
- **`Tally.Banking.StatementReader`** replaces `Tally.Banking.Metro`. The three documents in this archive disagree about labels, date formats, column order, which line the description sits on and whether the rows run forwards or backwards. **They agree on the two things that matter**: a running balance beside every row, and a stated closing figure
- **`Tally.Banking.Format`** is the behaviour each one implements, including `recognises?/1`, so a caller hands over text and gets an answer rather than choosing
- **`Format.Tide`** reads the Tide statement, which is the only document that can prove the Tide CSV exports. Its description sits on the line **above** its figures, and it labels each balance with its own date rather than as opening and closing, printing the closing one first
- **`Format.MetroExport`** reads the account's final month, which posted statements do not cover. Its rows run **newest first**: read in document order every direction would invert silently and the arithmetic would still close
- **`basis` records which checks actually ran.** `:stated` is three; `:rows_only` is none, and for those `Tally.Banking.chain/2` is the real evidence
- All 64 posted statements still parse unchanged, and the export's **derived opening balance of -£10.70 agrees exactly with April's closing figure**
Fixed
A Tide statement duplicated every transaction the CSV already held
- The Tide current account is imported from a CSV export, and a Tide statement covers one month of the same account. Importing both created each movement **twice** under different fingerprints, once from Tide's own transaction id and once from its position in the statement's table
- A statement row that matches an unproved row on account, date and amount now **promotes** it: the CSV row keeps its identity and its external id and gains the statement that evidences it, plus the running balance a CSV never carries. A row with no counterpart is created as usual
- That is what a statement is *for* here. The CSV exists because nothing else records that account; the statement exists to prove a month of it
- `Transaction.prove/1` is the resource's only update and accepts exactly two fields. **Nothing the bank stated can be edited** — not the date, the amount, the description or the external id
The Metro export's header balance is not a closing balance
- `Balance 0.00` at the head of the online export is the account's balance **now**, after it was settled, not the month's closing figure. May ended at -£9.82. Reading it as closing disagreed by exactly the £9.82 later paid, and the reconciliation caught it
Reconciliation compared the wrong two figures for any account still moving
- `Ledger.reconcile/2` compared the ledger's **whole** balance against the **last** statement's closing figure. Correct for Metro, where the statements run to the end; wrong for Tide, which has one statement covering March and CSV rows running to June
- It would have reported three months of real transactions as a reconciliation failure. Now compared as at the statement's own end date
Changed
Model routing is verified rather than guessed
- `google/gemini-2.5-flash` and `anthropic/claude-sonnet-4.5` both **checked against OpenRouter's live catalogue**, including that each accepts image input, with pricing recorded
- **No free model is usable**, and the privacy decision rather than capability is why. Of 428 models, 19 are zero-priced and 10 accept images; the only capable ones are served by Google AI Studio's free tier, which trains on prompts
- `docs/decisions.md` now states what the risk actually is, including where it is smaller than the rule implies, so the policy is not followed on faith
`Tally.Ledger`: five years of bank history as double entry, reconciled to the penny
- `Account`, `Entry` and `EntryLine`, **append-only and enforced by there being no other action**. A read and a create on each, no update, no destroy. A correction is a reversing entry, and `test/tally/ledger_test.exs` asserts the action list by name because this is the promise rather than an implementation detail
- **A line is one signed integer**, positive debit and negative credit. Two columns would permit a line with a figure in both, or in neither, and a ledger has no meaning for either. The balance check runs on the **argument**, before the entry or any line exists, so an unbalanced entry is never half-written
- **878 entries posted, trial balance exactly £0.00**, and the line that matters: **`Metro Bank Start Up Account: ledger and bank agree at -£10.70`**. 831 transactions across five years and four months, and the double-entry balance equals the bank's own closing balance
Classification has five tiers, reported separately because they are not equally good
- **12 transfers** between two accounts Tally holds, detected as opposite-signed pairs. Two bank statements agreeing, so no archive is needed. Posted as **one** entry: two would credit one bank and debit the other twice over
- **729 matched** against a Crunch movement, on exact amount within a window that varies by what the workbook's date means — an expense is dated by its invoice and paid weeks later
- **19 by label**, the bank describing its own charge
- **74 by precedent**, learned from the archive rather than configured: a description seen at least five times among matched rows and mapped to one account **every** time. It exists because the archive stops at 2026-01-14 and the statements run to 2026-05
- **13 by reference**, the weakest tier. `ref: Dividend` posts to the **director's loan account, not to Dividends**: the reference evidences that money reached the director and not that a dividend was declared, and the two are taxed differently
- **30 in suspense**, down from 150. Suspense keeps the books balanced and the ignorance counted; a guess would be a wrong figure that looks right, and omitting the row would break the only check worth having
Changed
The test suite runs in half the time, by fixing three modules that need not have been serial
- **16.0s to 8.9s** for 883 tests, stable across seeds. Three modules were `async: false` because they touch the document store, which is one directory the whole run shares
- **Content addressing makes concurrent writes safe**: two modules writing identical bytes produce one file rather than a collision. Only a module that **deletes or overwrites** a blob is unsafe, and exactly two do. Both stay sync and now say why, so nobody flips them for the speed
- The 30s figure in `mix quality` is coverage instrumentation rather than the tests. Partitioning was considered and rejected: `MIX_TEST_PARTITION` is already wired into `config/test.exs`, but it splits work across operating system processes and buys wall-clock only in a CI matrix. It earns its place on SQLite, where a single writer lock makes async tests contend on the database itself; Postgres does not have that problem and the contention here was the filesystem
Fixed
Classification depended on database row ids, so two installs disagreed
- **Development classified 74 rows by precedent and production classified 3, from identical inputs.** `internal_transfers/1` sorted by `transaction.id`, a uuid generated at import time, so the order everything downstream saw was a property of the database rather than of the data
- Now ordered by date, amount, account name and fingerprint. **The fingerprint is derived from the source and the row's position in it**, so it is the same on every install
- The regression test took three attempts to become one, and each failure is worth knowing. Asserting aggregate counts passed with the bug in place, because two rows swapping which movement explains them leaves every total identical. Asserting a sorted list of classifications passed a quarter of the time, because sorting collapsed eight days into a multiset. **Keying by date and comparing per transaction catches it 10 times out of 10**, verified by reintroducing the bug
- The fixture had the same fault as the code: its fingerprints were an incrementing counter, so it reproduced the very order-dependence it was meant to detect
A posting option leaked into Ash, the same shape of fault as `:accept` before it
- `movements:` was read with `Keyword.get` and the rest of `opts` passed on, so Ash rejected it with `unknown options [:movements]`. `Tally.Documents.store/3` learned this once already when `:accept` leaked and failed a daily step **after** the work was done. Popped now, with the reason beside it
Tests could not reach the matching or precedent logic at all
- `Posting.run/2` read its movements from the archive, which is empty in test, so every posting test ran with `matched: 0` and neither the window, the consumption rule nor precedent was exercised
- `movements:` makes them injectable, and thirteen tests now cover them: exact and near matching, the per-kind window, one movement explaining one transaction rather than four, a tax posted by the tax the workbook names, and both halves of the precedent rule
The ledger's bank account was out by £7,483.28, and every entry in it was correct
- 878 entries, every one balanced, trial balance exactly zero — and **the opening balance of the oldest statement had never been posted**, so the ledger started at zero while the real account did not
- Found by building `Ledger.reconcile/2` and running it, which is the point: a per-entry check proves an entry is consistent and a trial balance proves the ledger is consistent. **Neither says the ledger describes the same money the bank does**
- Now posted against `3100 Balance brought forward`. An account with only a CSV export gets none, and reconciles as `nil` rather than zero, because **nil and zero are different claims**
Posting the same bank row twice doubled it
- `upsert?: true` on the entry's fingerprint returns the existing entry, which looks idempotent, and `manage_relationship` then writes its lines **again**. Same entry, twice the amounts
- Caught by a test asserting a balance after a second run: both runs reported success and every entry still balanced
- Now a plain create behind a unique index, with `Ledger.post_entry/2` looking first
`Tally.Gateway`, and a containment model that is now asserted rather than described
- **`Tally.Gateway.Behaviour` has exactly one callback**, `complete/3`. No tool calling, no function schema, no streaming hook. Tool use is not disabled by configuration; there is nowhere in the contract to pass one, and a test asserts the callback list
- **The Gateway's reachable modules are a whitelist read out of the compiled BEAM**: `Storage`, `OpenRouter`, `Prompts`, `Reading`, and nothing else. A blacklist naming `Integrations.Client` would have let the next outbound door in unnoticed
- **Model routing is config**, primary plus a fallback chain per job. A response failing schema validation re-routes to the next model, which is exactly what a cheap model that cannot hold a format needs. Failing on every model is a failed job
- **`Tally.Gateway.Reading`** validates every response through an Ecto changeset before a field is read. A response that fails is a **failed job, not a partial application**
The model transcribes; Elixir converts
- An amount comes back as `as_printed` — the literal text on the page — and `Tally.Money` computes the pence. **The model never returns a number.** A model may describe a calculation and may not perform one, and reading a figure off a page sits right on that line
- **An amount whose printed form does not parse is rejected**, which catches a figure invented in a plausible format that arithmetic would never have questioned
- **`unreadable` is required**, and it is the useful field. A model asked for a figure it cannot see will produce one; asking what it could not read gives it somewhere else to put that
Extraction produces a proposal, never a change
- **`Tally.Documents.Extraction`** is inert: no update action touches the reading, and `:applied` is reachable only through `:approved`. A person approves and deterministic Elixir applies
- **A failed send is a row.** The privacy log answers "what has left this machine", so an attempt that produced nothing usable is recorded with the models tried. A **refusal** records nothing, because nothing was sent
- **`Document.local_only`** is the owner's veto, enforced before any client is called. It is the condition attached to the privacy exception that permitted a raw page to reach a cloud model at all, since an image cannot be redacted before it is understood
- Hammox doubles the client in tests, checked against the behaviour's typespec, so the suite issues no request and cannot silently start
Fixed
An intermittent test failure that looked like a race in the ingest
- `:zip.create/3` stamps entries with the current time, so the same content zipped twice a second apart produces **different bytes and therefore a different bundle**. A test re-ingesting "the same archive" was sometimes ingesting a second one, and only when the suite ran slowly enough to cross a second boundary
- The fixture now stamps a fixed mtime. **The ingest is unchanged**: two archives made at different moments really are two archives
- A second one in the same area: `:code.which/1` returns `:cover_compiled` under coverage, so a test reading a module's BEAM passed alone and failed inside `mix quality`
Five years of bank history, and every row of it proved
- **`Tally.Banking`** with `Account`, `Statement` and `Transaction`. Statement-first on purpose: the statement holds **the bank's own** opening balance, closing balance and both totals, which is what makes its transactions checkable. Storing computed sums there would make the reconciliation circular and it would pass forever
- **`Tally.Banking.Metro`** reads a statement's text layer and returns nothing it cannot prove. Every row's amount must equal the movement in the running balance beside it, **and** the rows must sum to all three printed totals. A statement failing any check is refused whole, with the figures
- **Direction comes from the balance moving, never from a column position.** Ghostscript's text layer preserves spacing approximately and **the indentation changes between pages of the same statement**, so a column-position parser reads page one correctly and page two as money flowing the wrong way
- **64 of 64 statements reconciled, zero refused, 831 transactions, 2021-01-01 to 2026-04-30**, from 132 archived files: the nested `Metro Statements.zip` holds a second copy of every one, and the report counts distinct statements rather than files so it does not read as a doubling
- **`Tally.Banking.chain/2`** checks that consecutive statements meet on both period and balance. **The chain is unbroken across all 64.** A per-statement check catches a misread row; only the chain catches a *missing* statement, which is the fault that stays invisible for years
- The account's end is now legible: swept to zero on 2026-04-17 with £8,975.50 out to the new bank, then £10.70 of April fees taking it negative
An external audit of the parser, which it passed
- **`Tally.Banking.CrossCheck`** compares the Metro CSV export against the statements already imported for the same four months. Two independent renderings from different parts of the bank's systems, of which Tally had read only one
- **53 rows matched, zero differences either way, identical net movement of £8,435.01**
- Matched on date and signed amount, not description: the formats truncate differently, and comparing descriptions would bury a real difference under false ones. Compared as a **multiset**, because four £500 payments on one day are four rows
`Tally.Banking.Csv` and the Tide history
- Tide's two exports imported, 58 rows across a current account and an instant saver, with `statement_id` and `balance_pence` nil and **`Banking.unproved/1`** listing them. A CSV export carries no balances and no totals, so its rows cannot be proved complete and nothing pretends they can
- Only `Cleared` rows are taken. A pending transaction may not happen, and importing it would assert money moved
- The account comes from the file's own name, so the saver's rows cannot land in the current account
- Metro's export is `dd/mm/yyyy`. Read the other way, `02/10/2025` becomes February and lands four months from where it belongs, in a statement that would then fail to reconcile for no visible reason
Text layers for the whole archive
- **540 text derivations** recorded through `priv/tools/pdf_to_text.sh`, leaving **7 files unread**: six images that need a vision model, and one PDF that hung ghostscript
- `Archive.unread/1` no longer counts a text file or a ZIP as unread. Reading a text file is not an interpretation of it, and an archive's reading is the expansion that produced its items
- `Archive.derived/3` and `Archive.items_of_kind/2` are how an importer finds and reads its material, so `Tally.Crunch.read/2` is now one caller rather than the only way in
A changed manifest rule now corrects the documents it already created
- The archive was re-readable but not re-classifiable: a rule that filed a file wrongly produced a document row nothing would ever fix. `Archive.Item` now records `classified_kind`, **the kind the manifest gave a file when it was last ingested**, and a re-run compares it three ways
- **The document still agrees with it** — nobody has touched it, so the new rule corrects it through `reclassify_document/3`, and the audit log shows a correction rather than the value having always been that
- **The document disagrees** — a person reclassified it, and the manifest does not overwrite them. Reported as `overridden`. `Tally.Documents.Document`'s own rule, applied here
- **There is no record** (ingested before the field existed) — reported as `adopted`, kept separate from both. A nil is an absence of evidence about who chose the kind, and calling it a person's override would be a fabrication while calling it a rule change would claim a rule that never ran
The manifest was filing purchase receipts as sales invoices
- Crunch keeps expense receipts under `Receipt attachments/<expense id>/` and several are named `..._invoice.pdf`, so the generic "contains invoice" rule claimed them. **A supplier's invoice was being filed as one of ours: the wrong side of the books.** Nine files, four of them named as invoices
- Fixed by a rule placed **before** the generic one, which is the ordering guarantee the table form exists to make visible. Re-running the ingest corrected all nine in place
`Tally.Money`, in place of four private copies
- `Tally.Tax.HmrcRates`, `Tally.FreeAgent.Import` and `Tally.Crunch.Export` had each grown their own text-to-pence helper and the bank importers would have made a fourth. The arithmetic is identical and it is the arithmetic that matters
- Decimal throughout, never a float: `"7959.6"` times 100 in binary floating point is `795_959.9999999999`, and a penny is not a rounding detail when a statement either reconciles or does not
- Refuses rather than salvages. `Decimal.parse/1` returns `{1, "abc"}` for `"1abc"`, and accepting that turns a misparsed column into a confident figure
The archive is a domain, so the inbox can be re-read rather than parsed once
- **`Tally.Archive`** with three resources. `Bundle` is one archive, digest-verified, carrying `generated_on` and `regenerable`: the first because **every absence in the Crunch export has to be read against its 2026-01-14 cut-off**, the second because that account is closed and the export cannot be regenerated
- **`Item` is one file, with a disposition**, and the middle value is the point: `:archived`, `:excluded` **with a recorded reason**, or `:unsupported` where nothing recognised the bytes. Without that distinction, "this file has no financial content" and "nobody got to this file" look identical, which is the state `docs/status.md` was in when it asked a future session to remember one filename
- **`James_Greenwood_DBS.pdf` is now excluded by mechanism rather than by note**, and by digest as well as by name, so the same file arriving as `scan001.pdf` is still refused
- **`Derivation`** records a produced reading and the tool that produced it, with the tool's version being **its own digest** so it cannot go stale. A re-derivation supersedes rather than overwrites, keeping a figure recorded last month traceable to the reading that was current then
- **`Manifest`** is the whole policy as an ordered table of 22 rules, pure over the path. Reviewable in one screen and diffable when it changes
- **`Tally.Crunch.read/2`** is how importers read: workbook name in, current derivation's bytes out, digest-verified. Its three error cases are kept distinct because they need different responses. `:not_ingested` is a missing file, `:not_derived` is work to do, `:corrupt` is bytes that rotted
- **`Archive.unread/1`** answers the question the manifest exists for: which archived files nothing has yet derived a reading from. Excluded files are not unread, because a decision was taken
Ingested, verified, and idempotent on the real archives
- **574 files archived, 165 excluded, 0 unsupported, 0 problems**, across both bundles including the 65 statements inside the nested `Metro Statements.zip`. All 574 blobs verify against their digests
- **A second run creates nothing.** Same counts, no duplicate documents, no duplicate items
- **461 documents created**: 133 sales invoices, 131 bank statements, 92 payslips, 83 dividend vouchers, 83 sets of board minutes, 22 workbooks and the rest
- `received_on` is read from the filename where the source system put one there, so a 2018 dividend voucher is dated 2018 rather than the day it was ingested
Fixed
A leading newline made Tally refuse 65 real bank statements
- `Storage.accept/2` matched magic bytes at offset zero, and **every Metro Bank statement begins with a newline before `%PDF-`** while three carry **330 bytes of leading spaces**. All 65 were refused, and it looked like a rejected upload rather than a bug
- Now skips up to **1,024 bytes of leading whitespace**, which is the window PDF readers themselves scan for the header rather than a number picked to make a file pass. A bound guessed at 64 was tried first and still refused three statements ghostscript reads perfectly
- **The boundary is unchanged and tested as such**: only whitespace may precede the header, so an SVG or HTML file with a leading newline still begins with `<` and is still refused. The digest is taken over the original bytes, so what is stored is exactly what arrived
- This was a genuine upload bug, not only an ingest one, which is why the fix is in the strict list rather than confined to the archive door
The Crunch archive is readable, and it holds the dates FreeAgent could not evidence
- **`priv/tools/xls_to_csv.py`** converts the export's 22 legacy BIFF `.xls` workbooks to CSV. It interprets nothing: no column renamed, no row dropped, no number rounded. **The `.xls` stays the source of truth and the CSV is derived**, the discipline `Tally.Sources` applies to a published figure
- **`Tally.Crunch.Export`** parses the converted CSV. Money through `Decimal`, never a float: `7959.6` in binary floating point is `795,959.9999999999` and truncates a penny short of a figure that has to reconcile against a bank statement. A malformed row is **returned as a problem, never dropped**, because an importer that skips rows silently reports a clean run over incomplete data
- **`Tally.Crunch.VatReturns`** closes the gap `Tally.Vat.Returns` named in its own moduledoc. The five quarters held as filed-with-no-date now have real dates, and **two of them were late**: the quarter ending 2024-10-31 by **23 days** and 2025-04-30 by **18 days**. Tally knew about neither, which is the first law's weak half exactly — not a wrong figure, an absent one
- It **corrects a missing date and never overwrites a present one**. Where the archive disagrees with a FreeAgent-evidenced date the archive loses and the disagreement is reported: Crunch recorded sending the return, HMRC recorded receiving it, and only one of those is what a penalty turns on
- The `Period` column is the month the quarter **ended**, established three ways rather than assumed and corroborated by a screenshot nobody took for the purpose. Reading it backwards would have moved all 27 dates three months and looked entirely plausible. See `docs/decisions.md`
- **`Tally.Crunch.TaxPayments`** reads seven years of payments to HMRC. **33 payments, not one a penalty, surcharge or interest charge.** That raises the confidence of the eleven `settled_late` notices and closes none of them: the export ends 2026-01-14, one payment was made personally, and a charge appealed or never paid leaves no row at all
- **The wider finding, now visible for the first time:** of 25 VAT returns filed, **11 were filed late and 14 paid late**, the worst filed 170 days late and one paid **393 days** after its deadline. Against all of that there is no evidence of a single surcharge
The Time to Pay arrangement is readable, and it ends on 5 October 2026
- Found in `/mnt/raid/tally-inbox` as a GOV.UK screenshot: **£10,272.91 including £59.03 of interest**, reference `7794779634`, set up 2026-08-20, collected on the 3rd or next working day. **September's £5,136.45 is paid**; **October's £5,136.46 is the last one**
- **It is a two-instalment plan, not open-ended**, which reframes the risk this project kept circling: one payment left, and then the exposure ends. 3 October is a Saturday, so the obligation is recorded on **Monday 5 October**
- No HMRC API exposes a Time to Pay arrangement — verified twice — so the instalment dates could only ever come from the paperwork. **Both instalments are now obligations on production**, with the document and reference in the `source` field
- The full inbox is catalogued in `docs/status.md`: 421 files of the accountant's export and 102 of the owner's own, what each settles, and the finding that the `.xls` files are **genuine OLE2 legacy Excel** which nothing on this machine can currently read
A settlement date can now be unknown, or corrected
- **`Obligations.mark_filed_undated/3`** — filed, with `filed_on` left nil, which `Position.days_late/1` already declined to assess. A separate action rather than passing nil to `mark_filed`, because `set_new_attribute` treats an explicit nil as *not supplied* and silently records **today** — fabricating a filing date for an import of somebody else's history. **The note is required**: "filed, date unknown" is a claim about missing evidence, not a fact
- **`Obligations.correct_settlement_date/3`** — corrects `filed_on` or `paid_on` **without a state transition**, which is exactly why it was impossible before: the state machine rightly refuses `:filed` to `:filed`, and `update` does not accept the fields. `mark_filed`'s own comment anticipated a wrong date and offered no way back. It carries its own audit event kind, so the log shows a correction happened rather than the value having always been that
- A validation refuses `paid_on` on a filed obligation and vice versa: `delay_ended_on/1` reads the date matching the state, so such a correction would appear to have silently done nothing
- **`Tally.Vat.Returns` now records the five Crunch-era returns** as filed-with-no-date rather than refusing them. Holding nothing was the safer of two bad answers; holding the truth beats both
Interest, the part of a delay's cost that cannot be appealed
- `Tally.Interest.Rate`, `Tally.Interest.accrued/2` and `Tally.Interest.Seeding`, built to the shape decided in `docs/decisions.md`. **The real case prices at £310.57**: £18,515.19 of Corporation Tax due 2026-02-01 and paid 2026-04-21, 79 days at 7.75%, hand-checked against the arithmetic
- **`applies_to` names a published HMRC table, not a tax**, because HMRC publishes tables covering several taxes each plus separate ones for Corporation Tax by quarterly instalments (6.25% and 3.50% from 2025-12-29). `table_for/1` holds the mapping and a kind absent from it gets **no rate rather than the nearest-looking one**
- **A rate can change part way through a delay**, so `accrued/2` splits the period at every change and charges each stretch at the rate in force. A single-rate calculation would agree most of the time, which is what makes it dangerous
- **No history is seeded**, deliberately: a delay beginning before 2026-01-09 returns `{:error, {:no_rate, date}}` rather than being priced at today's rate or reported as nought
- `Tally.Obligations.LateSettlement` now reports **penalty and interest side by side rather than summed** — a penalty can be appealed on reasonable excuse and interest generally cannot, so a merged figure gives the reader the wrong answer to *what can I contest*
- **`total_is_floor?` stays true, and that is a correction.** It was briefly made conditional on the kind having an interest table, which would have reported a Companies House penalty as complete on the unverified assumption that such penalties never accrue interest. Guessing that way understates what a delay cost, which is the one direction that matters
"22 obligations are settled" was true and the most misleading sentence on the page
- `Summary.settled_late_count/1`, and the dashboard now separates settled from **settled after its deadline** — amber and the word together, linking to `/notices` because each one is an open question about what it cost rather than a row to admire
- **The obligations index already stated the margin, and had its own copy of the arithmetic.** It now asks `Position.days_late/1`: two implementations of "how late was it" agree right up until one is edited, and a screen quietly disagreeing with the notice about the same obligation is worse than either being wrong alone
- **Pinned a trap in `mark_filed`**: passing `filed_on: nil` silently records **today** rather than meaning *filed, date unknown*, because the action uses `set_new_attribute`. For a historical import that fabricates a filing date. `Tally.Vat.Returns` avoids it by never calling `mark_filed` without a real date; the action still offers no way to record an unknown one, which the Crunch import will want
The VAT history came from FreeAgent, and a filed return stopped showing as owed
- `Tally.Vat.Returns` imports the VAT returns FreeAgent can evidence. For returns filed through its MTD connection it holds `filed_at` **and `filed_reference`** — HMRC's own receipt number — so the `:vat_period_unrecorded` notice for the quarter ending 2026-04-30 now resolves because the period is genuinely recorded rather than because somebody cleared a banner
- **A live return that had already been filed stayed open forever.** The return due 2026-09-07 was showing as owed while HMRC had had it since 2026-08-17, filed 21 days early: `Vat.Obligations` creates the live quarter as `:open` and nothing ever told Tally otherwise. `plan/3` now separates *held and open* from *held and settled* and marks the first filed
- **Three deliberate refusals.** Five returns say `marked_as_filed` with no date or reference (the accountant's era) and nothing is created for them, because using `filing_due_on` as a stand-in would make a late filing read as on time. **No payment obligations at all** — every one of the ten payments reports `"unpaid"` including 2024 quarters certainly paid, because the field tracks bank reconciliation rather than whether HMRC has the money, and importing them would have asserted about **£38,000 of outstanding VAT**. And a cancelled obligation is never settled, only an open one
- **`:vat_period_unrecorded` could never resolve.** It was raised by `Tally.Vat.Obligations` and absent from `Refresh.resolvable_kinds`, so only a human could clear it. Its fingerprints now come from that step's own report, since it raises its notices itself rather than handing them back
- The judgements live in a pure `plan/3` so they are tested without stubbing an HTTP client, against the real payloads production returned
- Found on the way: the quarter ending 2025-10-31 was **filed 64 days late**, on the same day as the next quarter's early return
The filed accounts are readable now, not just archived
- `Tally.Filings.Ixbrl` parses inline XBRL into facts, and `Tally.Filings.read_accounts/1` reads an archived filing through it. Verified against GOLDSMOOR's real 2024 accounts before any test was written: 29 facts, the company number the document claims for itself, and a balance sheet — a property at £119,303 against £123,600 of creditors due after one year, leaving £237 of net assets
- **Contexts are resolved, not discarded.** That document reports `Creditors` **twice**, £504 and £123,600, differing only by a `MaturitiesOrExpirationPeriodsDimension` segment of `WithinOneYear` against `AfterOneYear`. Due within a year and due after it are different facts about a company, and a reader shown one number labelled "Creditors" has been actively misled
- Monetary facts come out as **integer pence**, with `scale` and `sign` applied — `scale="3"` means thousands, and ignoring it understates by a factor of a thousand in a way that looks plausible on a screen. `xsi:nil="true"` stays `nil`, because an empty line is not zero
- **`{:saxy, "~> 1.6"}`** rather than OTP's `xmerl`, on a security argument recorded in `docs/decisions.md`: these are documents from an external source and xmerl fetches external entities unless correctly told not to, where Saxy does not process them at all
- **The parser reports a filing's own tagging errors faithfully**, also recorded as a decision. GOLDSMOOR's accounts tag employee count with `unitRef="GBP"`; a test asserts Tally does not silently correct it
- Also added: `Storage.fetch_verified/1`, which confirms a blob still hashes to its name on the way out. **A figure parsed from a corrupted filing is indistinguishable from a real one**, which makes silent corruption worse here than a missing file
- One answer falls out immediately: **ATED almost certainly does not apply to GOLDSMOOR**, since it bites on company-held dwellings over £500,000 and the property is on the books at £119,303
Fixed
Inline XBRL had never once been archived, and the report said so accurately
- **Verified on production: 8 fetched of 8 attempted, 0 failed.** Eight years of tagged accounts archived where the count was zero — every accounts filing either company has ever made, 184 numeric facts, GOLDSMOOR's 2024 accounts included. The other 21 filings are confirmation statements and address changes, which have no iXBRL rendering and now report as such rather than as failures
- The same run confirmed two designed behaviours: the ten settled-late notices came back **still standing** rather than duplicating, and the `refresh_failed` notice from the broken document step **resolved itself** once the step succeeded
- **And the first version of the fix broke the whole document step.** `Documents.store/3` read `:accept` from its options and then passed the same list to Ash, which rejects options it does not know: the production run failed with `unknown options [:accept]` *after* the type check had passed. Popped rather than read now. `Storage.accept_archival/1` had tests; the function calling it did not — **testing a pure helper and not the seam it plugs into is how a green suite ships a broken job**, so `store/3` is now tested at the level that actually failed, including that the upload path still refuses XHTML
- `Tally.Documents.Storage.accept/2` allowlists PDF, PNG and JPEG **by magic number**, and inline XBRL is XHTML. So `CompaniesHouse.History.fetch_data_documents/1` fetched the tagged copy of every filing daily and had it **refused by Tally's own storage every single time since the function was written**. Verified on production: **0 of 29 filings had a data document.**
- **An earlier entry called this a wording fault. That was wrong** — the failures were real, and the claim elsewhere that "the accounts are genuine iXBRL, 51 tagged facts" came from an inspection done by hand in a session rather than from anything Tally ever held
- Fixed with `Storage.accept_archival/1` and an explicit `accept: :archival` on `Documents.store/3`. **A second door, not a wider one:** XHTML can carry a script, which is the whole reason the strict list exists, so the upload path is untouched and a test asserts it still refuses XHTML. Serving was already safe — every document goes out as an attachment with `nosniff`, a sandbox CSP, and `octet-stream` for anything outside the three types
A settled obligation could have been settled late, and Tally said nothing
- `Position.days_late/1` and `Position.settled_late?/1` — **finished with and finished on time are different facts**, and `settled?/1` conflating them kept a real cost invisible
- `Tally.Obligations.LateSettlement` turns them into notices, wired in as the daily refresh's last step because it reads the obligations the earlier steps just reconstructed
- **Ten found across the two companies, £1,575 of computed exposure**, including three consecutive late accounts years. `Penalties.Consecutive` is visibly working: one day late prices at £300 rather than £150, and 35 days at £750 rather than £375, because the scale doubles after a late year
- **Tested through `Notices.raise_notice/2`, not just as findings.** A finding map that the notices system refuses would surface as a crashed scheduled job rather than a failing test, so the join is covered: raising, seeing the same one again without duplicating it, and resolving once the charge is recorded
- **Each is a question, not a bill.** No penalty is recorded against any of them, so recording what was actually charged is what resolves the notice — dismissing it will not, because the question would still be unanswered. Kinds with no scale are reported without a figure: a late Corporation Tax payment carries **interest**, and £18,515.19 paid 79 days late is exposure `Tally.Interest.Rate` cannot yet quantify
Added
A second directorship nobody was watching, with accounts due in 23 days
- **Live on production**, and the reconstruction immediately earned its keep: **a probable £150 late filing penalty** on GOLDSMOOR's first accounts, due 2025-09-19 under the s.442(3) rule for a first period longer than twelve months and filed 2025-09-28. Nine days. Recorded as a question rather than a finding — the deadline is derived, and this codebase has already had one bug in that exact rule
- **Which exposed a real gap: a settled obligation can have been settled *late*, and Tally says nothing.** `Position.settled?/1` short-circuits it, so the penalty the reconstruction found appears nowhere. Every other piece exists — `delay_ended_on/1` returns the filing date, `project/2` prices it at £150.00, `still_running?` correctly reports false. What is missing is a notice kind and a screen that separates *settled* from *settled late*
- **`:unsupported_type` is being counted as a document fetch failure.** All eight "failures" in the production run are confirmation statements and address changes, which never have an iXBRL copy, while the "no machine-readable copy" column reads zero. `needs_attention?/1` is unaffected so no spurious alerts — but a line reading *8 failed* every morning is how a person learns to skip it
- `Tally.Entities.Appointment` — a person holding office at a company, with the role and the appointment date from the public register and nothing else: no dates of birth, no residential addresses. Upserts on `(person, company, role, appointed_on)` so re-reading a public officer list cannot double-count an exposure, and **resigned rows are kept rather than deleted**, because liability for a period does not end when the appointment does
- `Entity.control` — `:direct` or `:shared`, **recorded rather than derived**. It decides what a notice is allowed to say: a `:shared` entity's obligation cannot be instructed, only asked about
- `Tally.Entities.Seeding` and `Tally.Release.seed_entities()` — creates the owner, both companies and both directorships, idempotently. It exists because `docs/status.md` records that every entity row on production was once created by hand in a terminal session that has since ended, and **a directorship is far too consequential to hold only in someone's shell history**
- **Seeding will not put back a `control` value a person has corrected**, and does not touch name or incorporation date, which Companies House sync already owns on a fill-if-nil basis. Two writers to one field is how a value starts flickering between runs
- `CompaniesHouse.Refresh` needed no change: it already iterates every company entity carrying a number, so the **daily 06:15 job picks up the second company with no further wiring**
- **GOLDSMOOR PROPERTY LTD (15360509)** recorded: the owner is one of three directors and one of three equal shareholders of a property company formed by his brothers, whose setup he says he does not understand. Read from the public register — active since 19 December 2023, SIC 68209, £3 of share capital, three directors all identity-verified, a clean filing history, and **accounts for the period ending 31 December 2025 due 30 September 2026**, where last year's were filed two days before the deadline
- **Why it matters more than his own company in one respect:** a director's duties are personal and do not transfer to whoever keeps the books. Late accounts are a civil penalty on the company — £150 to £1,500, doubled two years running, a scale Tally already holds — **and a criminal offence by the directors** under s.451 Companies Act 2006. He is exposed to a filing he cannot see, check or perform
- **`control` becomes a property of an entity** (`docs/decisions.md`): an obligation he cannot discharge needs a different action and a different tone from one he can, or Tally either nags him about something he is powerless to fix or stays silent about a criminal offence. The only true thing such a notice can say is *ask whether this is in hand*
- **Corrected an assumption in the stamp duty exposure recorded earlier today.** A dwelling held by a company is not held by its shareholders, so if the owner's only property connection runs through GOLDSMOOR he may still be a first-time buyer and the ~£25,000 may not be at risk. Recorded as a question worth resolving rather than an answer
- **GOV.UK One Login identity verification grants no data access.** It is the Companies House director-identity requirement under the Economic Crime and Corporate Transparency Act — the register shows all three directors verified, which is what it is for. HMRC authorises per taxpayer through that taxpayer's own Gateway, so **GOLDSMOOR's tax position is invisible to Tally**. The public register plus the owner's mailbox is the whole surface, and the filed micro-entity accounts carry a balance sheet the existing iXBRL code can already read
A published privacy notice and terms of use, at `/privacy` and `/terms`
- `TallyWeb.PolicyLive` serves both, **public** alongside `/about` and linked from the footer of every page — a policy behind a login is not a published policy, and HMRC's approvals process reviews a developer's publicly accessible content
- **The pages lean into being non-commercial rather than papering over it.** HMRC's terms of use scope the privacy requirement to *"commercial software … customer data"*, which Tally is not and has none of; the terms and conditions requirement is not so scoped. So both pages say plainly that there are no customers, that the software is not offered to the public, that it cannot file or move money, and that nothing derived is advice — **a better answer to a reviewer than boilerplate implying customers who do not exist**, since it explains why there is no pricing page or support desk
- **Every claim on them is a checkable fact about this codebase** — AES-256-GCM credential encryption, `restic` encrypting client-side so the backup provider holds ciphertext and no key, backups in the EU, read-only connections throughout — and the module documents the rule that they change in the same commit as the architecture. A reviewer who finds one stale has a reason to doubt the rest
- Six tests, asserting the property that actually matters: **both are reachable without signing in**, they render without a socket, and they carry the specific claims a reviewer looks for. Moving them into the authenticated scope by accident would break a production application without breaking anything a person would notice
- **HMRC's approvals process also checks the developer**: Companies House status Active, plus the company website and public footprint. The register check passes; the footprint is the soft spot for a one-person consultancy, and these pages are now part of it. There are also **six months to complete a production credentials request** once started
HMRC's API catalogue swept against every perspective, 2026-09-07
- **The HMRC authorisation grant lasts 18 months, which is a new failure mode for this codebase.** FreeAgent and Fastmail refresh indefinitely, so no connection here has ever had an expiry date. An unattended `BATCH_PROCESS_DIRECT` job cannot re-consent on its own, so at 18 months the daily check stops and the only symptom is that nothing gets reported — exactly the silent failure this project exists to prevent. The connection date must be recorded and the approaching expiry treated as an obligation that raises a notice ahead of time
- **The application's privacy policy and terms URLs are unset**, under a heading about customers needing to know how their data is used. There are no customers, so these may be waivable — but the Production Approvals Checklist is the wrong place to find out otherwise, so two short static pages before the production application is the cheap safe move
- **Nothing is lost by dropping View Self Assessment Account**, confirmed against the OpenAPI specification: Self Assessment Accounts (MTD) 4.0 is a strict superset — charge detail and history by transaction and by reference, `balance-and-transactions` (the overdue/payable/pending breakdown with due dates), `payments-and-allocations`, coding out, **and a `/penalties` endpoint the dropped API never advertised**
- **The interest fields in 4.0 change item 1's design.** Interest comes back **per charge** — `interestRate`, `accruedInterest`, `accruingInterestAmount`, `interestStartDate`/`EndDate`, `interestOutstandingAmount` — and `calculateAccruedInterest` is a query parameter, so HMRC will compute it on request. **`interestRate` is HMRC stating the rate it applied**, so for Self Assessment `Tally.Interest.Rate` becomes the cross-check rather than the authority. The decided rule shape is unchanged; Corporation Tax and VAT will rely on it while Self Assessment verifies it
- **The Time to Pay arrangement is not readable, but its footprint is.** `isInterestAccrualOnHold`, `isInterestChargeOnHold` and `interestDunningLockAmount` are what an arrangement sets on the charges it covers — a dunning lock being HMRC's own term for suspended collection. Better than this morning's conclusion, which assumed only the overdue total would ever be visible
- **View Self Assessment Account dropped; the sandbox list is thirteen.** Noticed missing from the subscription screen, and the reason is decisive: it is a real API (`self-assessment-liability-api`) reading exactly the right thing — liability broken into charges, payments, credits and **interest** — but **Sandbox only, with no production environment**, so it cannot carry the capability it was added as insurance for. It also gates on *requesting* access, which is why it is absent from the list, and its own service guide recommends pairing it with Initiate Payment, declined by decision. **Self Assessment Accounts (MTD) 4.0 therefore has no alternate** — it is the only route to this capability, and the first production request is that one version. A direct vindication of the version/environment trap recorded above: the environment label decides, not the name or the number
- **Recorded an unrecorded exposure raised in conversation: a share of a property nobody has valued.** The owner is on the deeds of a house his parents live in. It does **not** affect his partner's Lifetime ISA — that test is per account holder, and HMRC's conveyancer guidance is explicit that a LISA investor may buy jointly with non-first-time purchasers. It does remove **Stamp Duty first-time buyers' relief from an entire joint purchase**, and may add the 5% additional-property surcharge: illustratively £5,000 against £30,000 on a £400,000 purchase. Relief asks whether he has *ever* acquired an interest and so cannot be restored; the surcharge depends on ownership at completion and is at least potentially avoidable. A latent capital gains exposure exists on the share as well. The crux is whether he holds a *beneficial* interest or sits on the legal title as a nominee — a question for a solicitor reading the deeds. Recorded as an unquantified exposure only, because an oversight tool that silently omits a possible five-figure liability is worse than one that records it as uncertain
- **Recorded a version-selection trap: the newest version of an HMRC API is often sandbox-only.** Individual Benefits 2.1 is Sandbox only while 2.0 and 1.1 are Sandbox and Production, so "use the latest" would mean building something that can never ship. The filter that matters is **"Sandbox and Production" versus "Sandbox"**, not Beta versus Stable — nearly every MTD API is permanently Beta. `Self Assessment Accounts (MTD)` is at 4.0, Beta, Sandbox and Production. The version is selected per request by the `Accept` header (`application/vnd.hmrc.4.0+json`) rather than by the subscription, so holding several versions costs nothing and the header should be pinned explicitly
- **The minimum functionality standards question answered from HMRC's own service guides, and the first production request cut to a single API.** VAT (MTD)'s production approvals section names **retrieve obligations *and* submit VAT return** as the mandatory endpoints for minimum compliance; MTD Income Tax's equivalent standards mandate quarterly updates, annual finalisation, loss handling and a return, naming Business Details, BSAS, Losses and Calculations. **Self Assessment Accounts (MTD) appears in neither** — it is a standalone view of liability, and so the one API carrying the Time to Pay watch is also the one with the cleanest approval path. Nothing in either guide addresses read-only software at all, which is a real ambiguity rather than a gap in searching, so the plan **avoids the question instead of waiting on it**
- **Dropping VAT (MTD) from the first request costs little now.** Tally already reads VAT obligations and amounts from FreeAgent, which is HMRC-recognised software that does the submitting. VAT (MTD) would add authoritative period dates, payment confirmation and 24 months of penalties — real value, no urgency, and not worth building a submission path already decided against
- **Corrected: nothing on the list is access-restricted.** An earlier pass inferred that Individual Benefits, National Insurance and the rest were private APIs that would be refused, on the evidence that their documentation pages 404. The URLs had been guessed at version `1.0` and these APIs are published at `1.1` and `1.2`. **Business Details (MTD) 404ed the same way**, and that is unmistakably an ordinary public MTD API — a 404 on a guessed URL is evidence about the guess. Individual Benefits is **user-restricted**, meaning ordinary OAuth where an individual reads their own data, and its stated purpose is pre-populating a Self Assessment return
- **Corrected: the National Insurance API does not report qualifying years.** It returns class 1 total earnings and class 2 contributions due per tax year, so whether a year counts must be **derived** — earnings at or above the Lower Earnings Limit, or class 2 paid or credited — and labelled as derived. The authoritative source, *Check your State Pension forecast*, has no API: the same shape as Time to Pay and secure messages
- **The production request stays narrow for a better reason.** Not refusal risk but testing cost: HMRC requires every endpoint in the request be tested before the review begins, so each extra API is work standing between today and the Time to Pay watchdog going live
- **Individual Employment, Individual Income and Individual Tax dropped entirely.** Nothing is built on them, they are in no production request, and a sandbox subscription would buy only version-retirement notices for APIs nobody calls
- **Corrected: the production review cannot be started early.** Item 0 previously said to apply on day one so the ten working days would run while the integration was written. HMRC grants production access *after* being satisfied the endpoints have been tested and after seeing evidence of fraud prevention headers being sent accurately, via a Production Approvals Checklist, and SDST must be contacted **within two weeks of finishing testing** so the sandbox logs are still visible. **The ten-day review is the last step, not the first**, so production access is realistically four to six weeks out — which means Tally cannot watch the Time to Pay arrangement for at least a month and a manual reminder is needed in the interim
- **`Gov-Client-Connection-Method` decides which fraud prevention headers are required**, and `BATCH_PROCESS_DIRECT` exists for unattended scheduled work with no user present. Recorded the recommendation to route **every** HMRC call through the batch path and expose no button that calls HMRC directly: one connection method rather than two, and no browser device-detail plumbing through LiveView. It also matches what Tally is — a dashboard that says "as at 06:00", not an interactive HMRC client
- **National Insurance and Business Details (MTD) promoted** into the build-against tier. Business Details is the only source of the `businessId` every other MTD income-tax call needs
- **Two test-support companions were missed** and are now listed — National Insurance Test Support and Marriage Allowance Test Support. **View Self Assessment Account** reinstated in sandbox as insurance on the most important capability, since one of the two Self Assessment account APIs may be deprecated
- **The production request is split deliberately.** Four unrestricted APIs first; the five restricted ones as a separate later request, because a business-case conversation about them would be held while the Time to Pay watchdog waits
- All 62 subscribable secure-data APIs read against the company, the director and his partner, and the outcome recorded in `docs/status.md` with a final sandbox subscription list and a narrower production one
- **VAT (MTD) was missing from the plan and is the most valuable subscription available.** `/obligations`, `/liabilities`, `/payments` and `/penalties` are all readable, which turns `Tally.Vat.Periods` from the source of truth into the fallback for when HMRC is unreachable, and answers the open `vat_period_unrecorded` notice outright. 24 months of VAT penalties, of which Tally currently holds none, come with it
- **Marriage Allowance** recorded as a lead worth £252 a year, blocked on whether the director is a basic rate taxpayer — a fact Self Assessment Accounts (MTD) answers as a side effect of the work already planned, rather than by hand
- **A qualifying-income threshold watch** recorded as a small future item. The MTD for Income Tax test is **gross turnover before expenses** and each threshold is tested on a completed tax year, so the obligation is knowable a year ahead from a single number
- **The sandbox list cut from about forty APIs to thirteen.** Sandbox returns synthetic data for test users, so it proves OAuth, fraud prevention headers and response shapes and says nothing about whether an API is useful here. Holding the rest of the `*(MTD)` family as optionality was hedging against a scarcity that does not exist
- **National Insurance qualifying years** recorded, and they carry the sharpest deadline found so far: a gap can only be bought back for **six years**, after which it is permanent. Class 2 is £3.65/week against Class 3 at £18.40/week in 2026-27 — a fivefold difference hanging on whether someone is registered self-employed, which makes *paused* versus *ceased* worth roughly £767 a year on top of its MTD consequences
- **`/salary` should say whether a salary secures a qualifying year** and currently does not. Same class of gap as the Self Assessment one — Tally computing one side of a two-sided decision
- **Marriage Allowance has a ceiling**: net zero once the transferor's income passes the Personal Allowance, and it renews until cancelled. Recorded alongside the observation that MTD tests gross turnover while Marriage Allowance tests net profit, so the same business yields two different figures and a screen showing one for both would be wrong
- **Business Details (MTD)** reassessed upward. It is the only source of the `businessId` every other MTD income-tax call requires, and it carries HMRC's recorded commencement and cessation dates — the authoritative answer to *paused or ceased*
Decided
- **Tally watches money and does not move it.** HMRC's Initiate payment API is declined outright, including in sandbox, because a tool that notices mistakes and can also pay them has a strictly larger blast radius than one that can only notice. The same reasoning declines the legacy XML submission channels
- **Making Tax Digital for Income Tax cannot apply through the company.** HMRC's guidance excludes company salary and dividends from qualifying income by name, so this is closed rather than merely unlikely, and the `*(MTD)` income-tax family is inert for this entity
Slice 3: Companies House, read-only — the first connected system
- `Tally.CompaniesHouse` reads a company profile through the same archiving door as every published source, so a filing deadline is followable back to the payload it came from. Its Public Data API has **no write endpoints at all**, so the read-only declaration matches what the upstream can do rather than merely what we intend
- `Tally.CompaniesHouse.Sync` turns the two filing deadlines into obligations. A hand-entered obligation is **left alone and reported** where it disagrees, never overwritten
- **The shape was confirmed against the live API**, and that caught an assumption: a dissolved company carries an `accounts` block with **no `next_due` at all**, so treating the absent field as a broken payload would have errored on every closed company on the register
Eight years of history, reconstructed
- `Tally.Filings` records what was actually filed and when — historical fact, distinct from an obligation, which is a commitment. 22 filings imported from incorporation
- All 22 filed documents retrieved into `Tally.Documents`, content-addressed and backed up
- **The accounts are genuine iXBRL** — 51 tagged facts, 13 numeric — so a multi-year balance sheet is extractable deterministically. They are micro-entity accounts, so there is **no profit and loss in the filing at all**, which is recorded so nobody builds on an assumption it is there
- `Tally.CompaniesHouse.Reconstruct` creates the obligation that existed for each past period and marks it filed **on the date it was actually filed**. Obligations rather than a filings screen, because a filing record only exists if something *was* filed — a period never filed produces no record, and a screen built on filings is blind to exactly the case that matters
- `Obligation` gained `filed_on` and `paid_on`. The state machine recorded *that* something was filed and the audit log recorded *when somebody pressed the button*; for anything filed late and updated later those are different dates and only one is true
- The obligation page shows **what was filed**, with the document attached
Deadlines and penalties, computed from published rules
- `Tally.Deadlines` seeds five filing deadlines from GOV.UK. **Validated against the register before being trusted**, which caught something GOV.UK does not mention: naive month arithmetic makes a period ending 30 April due on 30 January, and Companies House publishes 31 January. The Companies Act's corresponding-date rule puts a month-end reference date on the month end — wrong by one day, on the wrong side of a penalty, on every month-end year end in the country
- The Companies House penalty scale, which forced two corrections: **bands do not accumulate** (£150 *or* £375 *or* £750, never their sum — summing would have reported £1,275 for a 39-day delay), and **months are not thirty days**
- **Computed exposure and a recorded charge are never merged.** Companies House penalties are not on the public register — verified — so a computed figure with no matching notice is the normal case, not an anomaly. Presenting it as a charge would invent a debt; presenting nothing would hide one
Added
Fastmail connected, over OAuth
- Fastmail no longer accepts app passwords for JMAP — bearer only. The same password authenticates over IMAP perfectly, which is what makes the failure confusing until you read the header
- Dynamic client registration accepts **loopback redirects only**, so authorisation happens once via `mix tally.fastmail.connect` and the refresh token is transferred to production
- **RFC 8707 `resource` is required**, and its absence fails at the redirect rather than at the request — invisible until a person has already walked through the consent screen
- The JMAP API URL is **discovered from the session** and is regional. Hardcoding the well-known host would work for the session endpoint and fail for everything else
- Two bugs in the connect task, both found by running it: a five-minute timeout that expired under exactly what it was waiting for, and an error clause below the state-mismatch clause — an OAuth refusal arrives *with* a state, so every real error would have been reported as a state mismatch
VAT: an eight-year blind spot, closed
- **The company has been VAT registered since 2019-05-07 and Tally held not one VAT obligation.** Thirty-two quarters, none of them visible. Found by connecting FreeAgent and reading `sales_tax_registration_status` — not by anything Tally noticed on its own, which is exactly the failure the first law names
- **`Tally.Vat.Registration`** is a resource rather than columns on the entity, because a registration has a history: schemes are joined and left, bases change, businesses deregister. Flattening it would keep only the current answer, which is the wrong one for every earlier period
- **`Tally.Vat.Periods`** derives every period from the registration's own anchor, so a disconnected FreeAgent cannot make VAT disappear. Each period end is computed as `anchor + 3n months` rather than by adding three months to the one before — that drifts permanently the first time a quarter clamps to a shorter month, and each individual step looks reasonable while the sequence is wrong
- **Only live periods become obligations.** Thirty-one of the thirty-two are settled history; creating them would assert eight years of unpaid VAT and bury the quarter that is genuinely live. The most recent unaccounted-for period becomes **one** notice — a question, not a debt, and not one per quarter since 2019
- Return and payment stay separate obligations. They share a deadline and nothing else: late submission costs points, late payment costs a percentage plus interest
A day lost in the date arithmetic, found by reconciliation
- **`Deadlines.apply_rule/2` applied the month-end adjustment after the day offset**, so `end_of_month` swallowed the extra day. Silent, because it only shows where a rule has *both* a month offset and a day offset — which Corporation Tax does: nine months **and one day**. For a period ending 30 April it gave 31 January where the answer is 1 February
- Caught by FreeAgent's tax timeline disagreeing with Tally's derivation by exactly one day. VAT has the same shape — one month and seven days — and an April quarter end was coming out a day early for the same reason
- The seeded Corporation Tax payment rule also had `month_end_sensitive?: false` where it should be true. Both fixed, and verified against six known-correct dates from two independent sources
FreeAgent, connected read-only, with the tokens encrypted at rest
- **`Tally.Credentials`** stores third-party tokens as ciphertext. A FreeAgent refresh token grants read access to the whole of a company's books and a Fastmail app password to years of correspondence; the restic backup being encrypted protects the copy in Backblaze, not this database, a `pg_dump`, or anything that can reach Postgres. AES-256-GCM through OTP's own `:crypto`, keyed by `TALLY_CREDENTIAL_KEY`. **Losing the key costs a reconnection, not data** — deliberately, so nothing irreplaceable may ever be stored behind it
- Decryption is an **explicit call**, so a screen, a log line, an `inspect/1` or an audit payload cannot leak a token by accident. One row per provider: reconnecting replaces rather than accumulating, because five stale tokens is a table where nobody can say which is live
- **`Tally.FreeAgent.OAuth`** does the authorization code flow. Tally never sees the FreeAgent password. `state` is generated, held in the session, checked once and deleted, so a callback that did not originate here is refused rather than exchanged
- **`Client.authenticate/3`** is a narrow, declared exception to the write guard. `authorise/3` refuses every non-GET to a read-only integration and should — but a token exchange alters no record, and refusing it would push OAuth outside the single outbound door entirely. It posts only to the path the integration's config declares as `:auth_endpoint`, which is not a caller argument
- **Read-only is enforced here, not granted by the provider.** FreeAgent offers no read-only OAuth scope, so a granted token can write and `Client.authorise/3` is what stops Tally using it that way. `/integrations` says so rather than implying the provider limited us
- The credentials were **already wired through `compose.yml` and never read by the application**. Now read in every environment, because a credential that only exists in the container is one nobody can exercise before deploying
Three things the audit found, cleared
- **The dashboard headline said £0.00 when it meant "not known".** Every live obligation was a filing with no amount, so a total of nothing rendered in large type as owing nothing. `Position.total_owed_pence/2` already refuses to return zero for exactly this reason; the dashboard was doing the thing that module refuses to do. It now shows an em dash and says why, and reports how many of the total's obligations carry no figure. Corporation Tax made it worse: a real liability with a knowable deadline and an amount nobody can price until the accounts are done
- **The "no penalty schedule" message was stale and became wrong.** It said Companies House penalties "are not loaded here", which stopped being true the day they were seeded — and was never true of the confirmation statement, the only kind that reached it. Deriving Corporation Tax obligations made it reachable for CT too, where it would have been nonsense. Each kind now says what is actually the case: no tariff at all for a confirmation statement (the sanction is prosecution and being struck off), interest rather than a penalty for late Corporation Tax, an unseeded schedule for PAYE
- **`Filings.late?/2` removed.** Written when no deadline rule was seeded and the honest answer was always `:unknown`, it was superseded by `Deadlines.verdict/2`, which looks the rule up rather than making every caller find it. Two answers to one question is how they drift apart. Its moduledoc still claimed the rules were not seeded
Corporation Tax deadlines, derived because nothing publishes them
- **`Tally.Obligations.CorporationTax`** derives the CT return and payment from the accounting period. Confirmed against the live HMRC catalogue: the Obligations (MTD) API covers self-employment, property and income tax only, and Corporation Tax Online is an XML API for *submitting* a CT600. **No API returns CT deadlines**, and MTD for Corporation Tax was shelved — so "wait for the API" meant waiting indefinitely while the company's largest money obligation stayed absent
- **Guarded on period length.** An HMRC accounting period cannot exceed 12 months; where a Companies House period is longer HMRC splits it, and the first of the two falls **earlier** than anything derived from the period end. A period over 12 months therefore derives nothing and raises a notice. Not hypothetical: this company's first period was 12 months and 14 days
- Derived only for the period still open. Seven `:open` CT obligations for seven filed years would assert seven years of unpaid tax, which is not something Tally knows
The first set of accounts is not due nine months after the period end
- Companies House allows **21 months from registration** for a first set of accounts. Both rules were seeded; only the ordinary one was ever used, so every reconstructed first period carried a deadline computed with the wrong rule — here 2020-01-31 where it was really 2020-01-16, fifteen days **later** than the truth. Moot only because the accounts were filed five months early
- A first filing is recognised as one only when it is the earliest accounts filing held *and* falls within two years of registration, so a partial import does not misapply the rule
- **The sync now keeps the incorporation date it already fetched.** It read `date_of_creation` and discarded it, so a newly added company would silently get the wrong first-accounts rule. Filled only where nothing is held — never overwriting
The machine-readable accounts, finally fetched
- Companies House serves accounts as both `application/pdf` and `application/xhtml+xml`. Tally fetched only the PDF for the whole life of the feature, so seven sets of accounts sat in the store as pictures of numbers while a copy with the figures **tagged** was available behind the same API key. `History.fetch_data_documents/1` retrieves it, and a filing now carries both
- **Archived, not parsed.** Inline XBRL needs an XML parser this project has not chosen, and that choice belongs with the accountant and bank archives rather than smuggled in here
The Companies House read is now scheduled, and what it refuses to do reaches a person
- **`Tally.CompaniesHouse.Refresh`** — the entrypoint the schedule calls: deadlines, then filings, then documents, then reconstruction, each independent so one unreachable endpoint does not abandon the rest. Runs daily at 06:15 via `~/server-backup/tally-companies-house-refresh.sh`
- **An unattended run may make a deadline more urgent by itself and never less.** The harms are not symmetrical: a deadline shown *earlier* than it really is costs an unnecessary early filing, one shown *later* costs a missed filing and a penalty. So `Sync.decide/3` applies a date the register has moved **earlier** and defers one moved **later**, whatever the register says. `apply: :all` is the same path with a person in it, and is what accepting a notice runs
- **`Tally.Notices`** — a fact awaiting a person, held until they deal with it. Deliberately not slice 6's `Suggestion`: a notice is observed, not proposed, and no model is involved. Raising is idempotent on a fingerprint, so a conflict standing for three weeks is one row seen twenty-one times rather than twenty-one rows
- **A notice resolves itself** when a later run no longer finds the disagreement — but **only for the steps that actually completed**. A run that could not reach the register produces no findings, and reading that as "everything is fixed" would clear the board at the moment it is least earned
- **`/notices`, and a count in the header of every screen.** A structural test fails the build if a screen renders the layout without it, because a badge that is silently missing is the failure the whole mechanism exists to prevent
Fixed
- **The dashboard counted eight years of finished filings as still owed.** It was written when there were two obligations and first met seventeen when the history landed. `owed/3`, `next_action/2` and `timeline/2` each tested `state in [:open, :filed]`, which does not know that a set of accounts is discharged by being filed — so "owed over the next 12 months" said seventeen obligations when two were live, penalties attached to settled filings were summed into the total, and a list captioned "everything unsettled" held fifteen completed ones. All three now go through the single predicate `Tally.Obligations.Position.settled?/1`
- The `overdue?` calculation on the resource carried the same mistake in SQL, unused by any screen but waiting for the first one to reach for it
- **A late filing kept getting later after it was filed.** `Tally.Penalties.project/2` measured against today, so accounts filed 26 days late in 2019 were treated as seven years late. In a banded regime that walks straight to the top band: **£1,500 reported for a filing that cost £150**, with steps listed as still to come that filing had already avoided. The measurement date now comes from `Position.delay_ended_on/1`, and the rules in effect are read as at that date too, because the schedule that applies to a 2019 filing is the one published in 2019
- A settled obligation with **no recorded date** now returns `{:error, :undated}` rather than being measured against today. The gap in the record is a fact; a delay that grows every time the page is opened is not
- **Filing now settles a filing obligation.** `standing/2` treated only `:paid` and `:cancelled` as settled, so every filed return with a past deadline reported as **overdue** — fifteen finished periods rendered as a crisis, burying the two obligations that were actually live. What settles an obligation depends on its kind: a return is discharged by filing, a payment only by paying
- A settled row showed no filing date, so a filing 39 days late gave no hint it was late
- Filed documents were linked to their filing and **not to the obligation that filing discharged**
- A provenance URL that **404s**: `Tally.Sources` hardcoded `gov.uk` for every source, and Companies House serves its register from a different host. A citation that looks checkable and is not is worse than no citation
- A first archive was reported as **"has changed since it was last archived"**. It had not changed; it had never been fetched. `:new` and `:changed` are now distinct
- The obligation page's settled label said **"Paid"** for anything settled. A set of accounts is never paid
The source archive: every figure traces to the document it came from
- `Tally.Sources` keeps the **payload as fetched, byte for byte**, under its own SHA-256, in the same content-addressed store as uploaded documents. Citing a URL was a claim; this is the evidence, and it survives GOV.UK revising or withdrawing the page
- All 132 seeded rules — 115 tax, 17 penalty — link to the snapshot they were read from
- **One row per distinct content, not per fetch.** An unchanged page updates `last_confirmed_at`; a changed one creates a new row, so the arrival of a row *is* the change signal and each source keeps its full history
- **`mix tally.sources.check`** re-fetches all 21 sources and compares extracted figures against what is stored. A moved figure names the key, shows both values, and exits non-zero
- **It writes no rules.** A figure that moved is a decision, not a fact to absorb — it may be a revision, a correction, or a page restructured in a way the extractor read wrongly, and those want different responses from a person who has looked
- **Scheduled weekly**, running `bin/tally eval 'Tally.Release.check_sources()'` in the container. Cron mails on a non-zero exit
- **`/sources`** shows every document, its published and last-confirmed dates, whether it has gone stale, its earlier versions, and a download of the archived copy — served with the same headers as an uploaded document, because third-party JSON rendered inline is a script-delivery vector too
The penalty regime, and what it costs to leave it
- `Tally.Penalties` — five statutory schedules seeded from GOV.UK: Self Assessment filing and payment, Corporation Tax filing, VAT submission and VAT payment. 17 steps
- **A penalty is a schedule, not a value**, which is why `PenaltyRule` is not a `TaxRule`. Everything else about it is identical: the same four provenance fields, the same effective dating, the same refusal to guess when a rule cannot be found
- **Extraction confirms rather than discovers.** The shape lives in code and seeding checks the page still says what each step claims, so a moved figure or a restructured page stops the task naming every phrase that no longer matches. It will not silently produce a schedule with a step missing, which would understate what being late costs
- **The projection separates charged, upcoming and possible.** Upcoming steps carry the date they land — "£300 on 31 January 2027" rather than "it will get worse". History-dependent steps are named and never totalled: a third consecutive late Corporation Tax return costs £1,000 rather than £200, and one obligation cannot know whether that applies
- **Every total is a floor and says so**, because interest tracks the Bank of England base rate and is not seeded
- The obligation screen distinguishes three things that read identically as a blank total and mean opposite things: no HMRC schedule exists for this kind, the schedule exists and is not loaded, and nothing is owed
Fixed
- A penalty step with a zero trigger was reaching on the **due date itself**, so the screen would have shown £100 owed on the day something was due, before it was late at all
The education layer
- `Tally.Glossary` — one explanation per domain concept, referred to by key: `<.explain key={:qualifying_year} />`. `docs/design.md` requires that the same question gets the same answer every time it is asked, and prose written into each template can only match by coincidence — it stops matching the first time one copy is improved
- **No entry may contain a figure, and a test enforces it.** No amount, no rate, no threshold. Prose is the most inviting place in the codebase for a remembered number to reappear, and "the personal allowance is £12,570" reads as helpful right up until the year turns. Figures come from `TaxRule` and are rendered by the screen beside the explanation; entries explain mechanism
- **The obligation screens now explain their own vocabulary.** They previously explained none of it, which is where a reader first meets *obligation*, *penalty* and *period* — none of which mean quite what they sound like. The show screen picks its explanations by obligation kind, so a VAT return does not explain corporation tax
- **Two of the best explanations existed only on the component gallery.** Why the tax year starts on 6 April, and why VAT was never your money, were written as demos and reachable nowhere else
- A missing key **raises where the template asks for it**, naming the keys that exist, rather than rendering an empty disclosure nobody notices is empty
- `TallyWeb.EducationLayerTest` asserts the pattern against real pages: the same concept worded identically on two screens, no disclosure open by default, and the whole thing working without JavaScript
Changed
- The salary comparison's explanations moved **outside** the guard that hides the figures. The mechanism does not depend on this year's rates being seeded, and a reader who arrives when they are not still benefits from understanding the question
Documents
- `/documents` — upload a letter, notice, statement or forecast and keep it as evidence. `Tally.Documents`
- **Files on disk, not rows in the database.** The backup decision made concrete: the database is dumped hourly and is mostly re-derivable from FreeAgent, HMRC and the mailbox, while scans are the irreplaceable part and are backed up as files. `TALLY_DOCUMENT_ROOT` raises on boot rather than defaulting, because a silent default would put exactly that material somewhere nobody backs up
- **Stored under the SHA-256 of the contents, never under the supplied filename.** A path from a browser is structurally impossible rather than escaped, the same file twice is one blob and two rows, and corruption is detectable — `verify_all/0` re-reads every blob and compares
- **The type comes from the file's magic bytes**, not from what the browser claimed. PDF, PNG or JPEG only: an SVG or HTML file would be a script running on Tally's own origin the moment anybody opened it, and it needs no attacker for that to matter
- **Downloads are served deliberately, never as static files** — always `attachment`, `nosniff`, a sandbox CSP, and a content type chosen from literals. Behind the login, with a test asserting it
- **It reads nothing, and says so.** No extraction, no classification, no dates lifted off a letter. That is slice 5, and the page names what it cannot do rather than implying a smarter system
- Asks when a document **arrived**, not when it was scanned. A letter dealt with six months late is the case Tally exists for, and defaulting to today would erase the gap that matters
Qualifying years held
- **Tracking someone with no connection to the company.** A person need not be a director, an employee or a shareholder — a qualifying-years count belongs to a person, not to a role. The screen hangs off the position rather than the salary comparison, and says so
- **How years are earned without a salary**, because that is the case for anyone at home: National Insurance credits count toward the State Pension exactly as contributions do, and a forecast count already includes them
- **The Child Benefit registration, flagged prominently.** A parent registered for Child Benefit for a child under 12 gets Class 3 credits automatically *even if they choose not to receive the payments* — so a household that opted out over the High Income Child Benefit Charge and never registered loses a qualifying year for every year at home, with nothing anywhere announcing it
- **The transfer route between partners**, where the registered parent already has a qualifying year from work: the credits may be accruing to whoever needs them least
- It does **not** assess anyone's eligibility. The list is what GOV.UK publishes, and whether a particular year was earned is answered by the National Insurance record itself
- `/pension` — records what a State Pension forecast said, and every reading taken before it. `Tally.Pensions.StatePensionRecord`
- **The one figure in slice 1 that no source can supply.** HMRC publishes no forecast or qualifying-years API, and automating a login to the citizen-facing service was considered and rejected under the first law, so this is typed — with what was being read and the day it was read
- **Readings are append-only.** There is no update action: a correction is a new reading. A salary decision gets made against one of these, and "what did we believe when we decided that?" has to stay answerable afterwards
- **A reading goes stale after a year** and says so, because a qualifying year is earned every April and an older figure understates the position by at least one
- **An estimate is stored as an estimate and displayed as one**, rather than being quietly promoted to a figure alongside a real forecast
- **Per person, never combined.** Two directors hold different counts, and a household figure would answer the wrong question for at least one of them
- `/salary` now says what a count means for the decision: at the full rate it states plainly that another qualifying year adds nothing, so the pension column no longer separates the routes
- **A shortfall is presented as a floor**, because a record contracted out before 2016 usually needs more than the published count and that is not knowable here
- Three figures seeded from GOV.UK's *The new State Pension*: the years needed for the full rate, the minimum for any pension at all, and the full weekly rate. `TaxRule` gained a `:years` unit — a count is neither money nor a rate, and storing 35 as pence to fit an existing unit would mean every reader has to know it is a lie
The pension route
- An **employer pension contribution** as a third remuneration route on `/salary`, deductible against corporation tax like salary but attracting National Insurance on neither side and not taxed as income now
- **Bounded by the annual allowance rather than by pay.** A contribution made personally only gets relief up to your earnings for the year; an employer contribution is a company expense and is not limited that way, which is why a low-salary director can put far more in through the company than personally. The page explains the asymmetry rather than assuming it is known
- **"You keep now" and "Into pension" are separate columns and are never added.** A pound locked until minimum pension age is not a pound in an account, and a combined total would read as though it were. `Tally.Tax.Remuneration` enforces this: `net_personal_pence` never includes the pension
- **No route is marked cheapest once a contribution is entered**, because a contribution always lowers total tax — bolding the lowest figure would amount to recommending that money be locked away
- Five pension figures seeded from GOV.UK's *Tax on your private pension contributions*: the annual allowance, both taper thresholds, the lump sum allowance and the tax-free lump sum rate. All five are in prose across two parts of the guide, so `Tally.Tax.HmrcCurrentRates` now concatenates parts
- **What a pension costs to take out again is not modelled**, because it depends on growth and on marginal rates that do not exist yet. The rule as published today is stated instead
Fixed
- **`/salary` showed four dividends the company could not have paid.** Each route treated everything left after salary as distributable, ignoring employer NI and corporation tax entirely — at £60,000 profit it showed £52,522 arriving out of £60,000, and every one of the four routes tripped the "not affordable" warning below the table. Dividends are now derived from `Tally.Tax.Remuneration.distributable/2`, so each row reconciles exactly to the profit and the warning is unreachable rather than permanent
- **The page quoted a tax threshold that is not seeded.** `Tally.Tax.Remuneration` deliberately refuses to apply the personal allowance taper because the threshold is not among the seeded rules; the page printed the figure from memory anyway. It now names the rule without quoting a number, and a test asserts no unseeded figure returns
- `tax_rules.source_url` recorded the API path fetched rather than the page a person would open to check it. Provenance exists to be re-checked, and `/api/content/...` serves JSON
The salary comparison screen
- `/salary` — four routes side by side, what each costs, and whether the year still counts toward the State Pension
- **The four salary points are derived, not chosen.** Each is a threshold already in the seeded rules — nothing, the secondary threshold, the Lower Earnings Limit, the personal allowance — and each marks a point where something changes. If HMRC moves one, the route moves with it
- **It compares without recommending.** The lowest total tax is shown in bold and labelled as arithmetic rather than advice, with the note that a lost pension year is not recoverable by paying more tax later
- Two explanations expand in place: why a small salary still earns a pension year, and why a salary is cheaper for the company than a dividend
- **It names what it cannot account for** — qualifying years already held, the Employment Allowance, marginal relief, and any other income — rather than leaving a reader to discover that a complete-looking page was not
The salary route calculator
- `Tally.Tax.Remuneration` — employee and employer NI, income tax, dividend tax in the band the salary leaves, corporation tax relief, net personal income and total cost to the company. **Deterministic arithmetic over `TaxRule` and nothing else**: no constants in the file, no figures from memory
- Pure functions over a rules map, so the whole module is testable without a database. Every expected figure in the tests is worked out by hand in a comment beside it
- **It does not recommend**, per `docs/decisions.md`: compute and compare, never recommend
- **It does not apply the Employment Allowance.** A company whose only employee is a single director generally cannot claim it, and the rule has conditions this module cannot check. Applying it silently would understate employer NI — the direction that hurts
- **It does not compute marginal relief.** Between the corporation tax thresholds it returns the main rate and reports `:marginal`, so a caller can show an upper bound as an upper bound
- **It does not apply the personal allowance taper** above £100,000: that threshold is not among the seeded rules, and guessing it would be the mistake this project exists to prevent
- `qualifying_year?/2` turns on the Lower Earnings Limit rather than on whether NI was paid — which is the whole reason a small salary is worth taking, and what a dividends-only arrangement quietly gives up
Everything the salary comparison needs is now seeded
- `Tally.Tax.HmrcCurrentRates` reads GOV.UK's three current-rates guides: the personal allowance and band boundaries, the dividend allowance and rates, and the corporation tax rates with their marginal thresholds. **106 rules; 22 for 2026-27**
- **The dividend rates are 10.75, 35.75 and 39.35 per cent.** Worth stating, because from memory they would have been 8.75 and 33.75 — which is exactly the £6,500 mistake in a different suit, and exactly why nothing here is written from memory
- Those pages **do not say which tax year they describe**, so their figures are seeded only against the current year and never backfilled. A rate read today is today's rate; asserting it applied in 2022/23 would be a guess wearing a citation
- Two figures live in sentences rather than tables. That is still the published source read deterministically, with the sentence kept as the excerpt. What `docs/decisions.md` forbids is a *model* extracting a figure from prose, which is a different thing
The position page, and a header that reads as one thing
- `/` is the position rather than a redirect: what is owed over twelve months, how much of it is an estimate, how much is penalties, the single next thing to do, and the timeline underneath
- **The health score, time-to-green and reserve date are deliberately absent**, and the page says so. A score needs a model nobody has defined; both dates need to know what is coming in and what is already set aside, which is the ledger and allocation slice. A plausible score on the front page of a tax product is the most dangerous number in it, because it is the one acted on without being checked
- `Tally.Obligations.Summary` — the arithmetic, as pure functions over loaded rows with the reference date passed in. A month window clamps to the month end, so one month after 31 January is 28 February rather than 3 March
- The header was three siblings in a wrapping flex row with `ml-auto` on the last, so on a narrow screen the account block dropped to its own line pushed hard right and read as detached. The container was `items-baseline` while that block was `items-center`, so the two halves sat on different alignments. Now two groups, aligned the same way
Every component swept for dead classes
- `select`, `textarea` and `checkbox` had no borders because they still carried daisyUI's `select`, `textarea-error`, `checkbox-sm`. **These survived the earlier sweep because their class names are also HTML element names**, so they were dismissed as false positives
- The account menu was a bare `<ul class="menu menu-horizontal">` that `phx.gen.auth` injects into `root.html.heex`, above the layout entirely — unstyled, and outside the design system by construction. It now lives in the header
- `ObligationLive` and `AboutLive` never passed `current_scope` to the layout, so the header always rendered logged-out
- Swept exhaustively this time: every class token in `lib/tally_web` checked against Tailwind's utility shapes, leaving only heroicons and the five of our own. **`test/tally_web/no_daisyui_test.exs` now fails the build if one reappears** — generators will keep reintroducing them, and a daisyUI class renders as nothing at all: no error, no warning, just an unstyled element
The component library
- `TallyWeb.UI` — the product's own vocabulary, separate from Phoenix's generated set: `health_meter`, `next_action`, `figure`, `dated`, `pill`, `explain`, `provenance`, `not_yet`, `empty`, `callout`, `panel`, `breadcrumb`, `ledger`. Each carries the design rule it exists to keep, and each appears on `/design` in every state
- The **health meter** is the one the design document names as the trap: amber to cyan, never red to green, and never the gradient alone — a position, a number, a word and a trajectory, any one of which reads without the others. `role="meter"` with a value, so it is announced rather than shown
- **`test/tally_web/accessibility_test.exs`** asserts structure against the real rendered HTML of every page: duplicate ids, dangling `aria-labelledby`, uncaptioned tables, skipped heading levels, positive `tabindex`, images without `alt`, fixed pixel widths, one `<main>` and one `<h1>`. **Every check failed something the first time it ran**
- Fixed along the way: a hardcoded id that duplicated whenever a component appeared twice, a `~w` sigil silently relying on a non-breaking space to keep "At risk" as one label, and a table with no caption
A component library page, and forty dead class names
- `/design` — every component in every state on one page, behind authentication. Proposed after the flash message was found rendering as unstyled text in the corner of the screen
- **The cause was forty dead daisyUI class names across five files.** `toast`, `alert alert-info`, `btn btn-primary`, `fieldset`, `label` — daisyUI was removed from this project months earlier because its themes are built on a red/green axis, so every one of those rendered as nothing. Each was invisible until someone opened the screen that used it: the flash, the login page, the confirmation page, the form labels
- All forty replaced with tokens. A check now finds none anywhere in `lib/tally_web`
- The gallery has a smoke test, because a component library that silently stopped rendering would be worse than not having one
Forms, and an audit trail that names who acted
- `/entities/new` and `/obligations/new`. The application can be used without a console for the first time
- **`Tally.Audit.Changes.RecordEvent`** — an Ash change on the create and transition actions, so an event is written however the action is reached. Recording from the caller means every future caller has to remember, and the first one that forgets leaves a hole; a hole in an audit log is worse than no log, because a missing event starts to look like evidence the thing did not happen
- **An action with no actor fails.** `Tally.Audit.Event` requires an `actor_id` for anything that is not the system, so the raise is deliberate: "someone created this obligation" is not an audit record. A console or a job that genuinely is the system passes `actor: :system`
- `source` is required on the obligation form and is why it is not shorter. A deadline that came from "I think it's the 7th" is the failure this system exists to prevent
- `.input` gained a `sublabel`, because this product explains itself constantly and field guidance belongs in the component rather than in each form
Authentication
- `mix phx.gen.auth`, matching vaultarr and sprout. **Every route that touches the financial position now requires a session**; `/about` stays public because it says which build is deployed and nothing else
- **Registration exists once and closes behind itself.** A single-operator instance on a public hostname with an open `/users/register` is a back door onto the company's HMRC position, so registration is gated on there being no accounts at all. Adding a second person is a deliberate act from a console
- Chose `phx.gen.auth` over Traefik forward-auth deliberately. The event log requires an `actor_id` for every human action and the containment model depends on showing that a model proposed and a human approved — that needs identity, not a gate. Forward-auth would mean trusting `X-Forwarded-User` headers, a weaker guarantee than the application owning its own sessions, and there is no forward-auth infrastructure running to adopt
Slice 1: the obligation screen
- `/obligations` and `/obligations/:id`, built to `design/Obligation.dc.html`
- **The ordering is the advice.** `Position.by_urgency/2` puts the longest-overdue first, then nearest deadline, then settled — the top of the list is the next right thing
- **A duration accompanies every date.** "Overdue by 5 weeks · was due 2 Aug 2026", never the date alone, which would make the reader subtract. Approximate above a month, because "in 5 months" is what a person plans around and "in 148 days" is a number they then have to convert
- **No status is carried by colour alone.** Overdue is amber *and* the word; upcoming is cyan *and* its own wording. A status conveyed only by hue is one the primary user cannot read
- Estimates are labelled "estimated", exact figures "exact", and every figure names the record it came from
- **The page states what it cannot yet compute.** The artboard shows a penalty growing weekly and a further charge at six months; those are absent, because the statutory penalty regime is not seeded. A plausible growth figure would be the one thing this page promises not to do — and a test asserts the disclosure stays until the rules arrive
- `Tally.Obligations.Position` — pure functions over loaded data, taking the reference date as an argument so no test depends on when it ran
Slice 1: obligations and penalties
- `Tally.Obligations.Obligation` — one thing owed by a date. **Filing and paying are separate obligations**: a VAT quarter produces two, with different deadlines and different penalty regimes, and a return filed on time with the payment late is a different position from both being late
- **`due_on` is stored, not derived.** Statutory deadline rules are reference data like any other figure, and deriving "one month and seven days after period end" from memory is the same mistake as the £6,500 threshold. Each obligation records the source that stated its deadline
- `amount_is_estimate`, because a corporation tax provision and a VAT return total are not the same kind of number. A validation refuses the flag without an amount: claiming a precision qualifier for a figure that does not exist reads as "we have an estimate" when we have nothing
- **`overdue?` is calculated, never stored.** A stored flag is wrong every night between the deadline passing and something remembering to update it — precisely when it is being read
- State transitions are guarded by `AshStateMachine`: nothing returns to `:open`, and a settled obligation cannot be quietly reopened
- `Tally.Obligations.Penalty` — append-only like the event log. A cancelled or reduced penalty is a **new row superseding the original**, not an edit, so the sequence of charges and reliefs stays readable. Sign rules refuse a negative charge or a positive relief, both of which produce totals that add up and are wrong
- Penalties are **recorded, not computed**, until the penalty regime is seeded from HMRC. A wrong penalty figure gets quoted at an accountant or used to decide what to pay first
Slice 1: entities and the event log
- `Tally.Entities.Entity` — company, person or household in one table, because the product is a single view of how company money becomes personal money and modelling them separately would mean building the join twice. A validation refuses company-only fields on a person or household: a person carrying a company number is the sort of record that makes a later "which entity files this" query return something plausible and wrong
- `Tally.Audit.Event` — the append-only domain event log. **There is no update action and no destroy action**, so no code path, migration or console session can rewrite history; a test asserts their absence. Events are named after what happened in the business (`obligation.created`, `tax_rule.seeded`), not after which table was written, because the question this will be asked is "why did this suggestion appear" rather than "what did this row look like last Tuesday"
- **Models are first-class actors** in the log, alongside humans, workers and the system. The containment model depends on being able to show that a model proposed something and a human approved it, and a log that cannot tell those apart cannot evidence it
- `Tally.Audit.new_correlation_id/0`, threading one action through the jobs it spawns and the events that result. Indexed, because the table only ever grows
Slice 1: statutory reference data
- `Tally.Tax.TaxRule` — one statutory figure, one period, with its provenance. Money as integer pence, rates as basis points, so 1.85% employer NIC is exact rather than a float that is nearly right. Every row carries the source URL, the page revision it came from, the published table row and the date a human checked it
- `Tally.Tax.value!/2` and `rule!/2`, which **raise rather than fall back**. There is no nearest match and no default: a calculation that cannot find its rule stops. `Tally.Tax.RuleNotFound` says how to fix it and argues against the fallback the reader is about to add
- `Tally.Tax.HmrcRates`, which extracts figures from HMRC's published "Rates and thresholds for employers" guidance, fetched as JSON through `Tally.Integrations.Client`. **Checked 2026-09-05: HMRC has no reference-data API** — every API on the Developer Hub is transactional — so the published document is the source
- `mix tally.tax.seed`, seeding **all eleven tax years from 2016/17**, because the recovery position needs the thresholds that applied then, not today's. 50 rules seeded and verified
Foundations of judgement
- The three laws, ordered, at the top of `docs/architecture.md`. Do no harm including through inaction; obey and work toward the best long-term position, except where that conflicts with the first; protect the system, except where that conflicts with the first two
- `docs/status.md`, recording everything that lives outside the repository: the production compose service, Gitea access, which secrets are set, the CI base image, the design canvas
- `docs/roadmap.md`: twelve slices in dependency order, with an honest read on progress
- `docs/scenarios.md`: real situations that drive design, including the live PAYE penalties and the salary-versus-dividends question as it actually arose
- `mix tally.docs`, which fails when documentation has drifted from the code. Built because the living-documentation rule was only a rule, and this changelog had gone sixteen commits without being touched
Design
- Eight screens on a design canvas, and the design system they establish: a colourblind-safe cyan-against-amber palette, deep-space ground, monospaced tabular figures, the education layer, and the AuDHD information architecture
- Design tokens in `app.css`, replacing daisyUI
Changed
- **daisyUI removed.** Its themes are built on a red/green semantic axis, which is the one distinction this application's primary user cannot make. `core_components.ex` is restyled in place against the tokens. CSS is 5.9kB gzipped, down from 11.1kB
- **CI rebuilt around the base image.** Dependencies, the dialyzer PLT and the asset toolchain are baked in, so CI compiles the application and nothing else. 28 seconds, down from 251
- Test database uses WAL and a busy timeout after CI hit lock contention under parallel cases
Fixed
- `mix tally.budget.assets` measured the largest file in the assets directory, and `phx.digest` leaves previous builds behind, so a stale bundle was being reported as the current size. It measures by name now
- `Tally.Build.label/0` matched on a compile-time constant at runtime, leaving a branch that could never be taken. Found by dialyzer
Added
Foundation
- Phoenix 1.8 application on Elixir 1.19, with SQLite for both relational data and vector search
- `sqlite-vec` vendored per architecture and loaded on every connection by `Tally.Repo.init/2`
- Release tasks that migrate and seed without starting the full application
Safety model
- `Tally.Integrations.Client`, the single outbound door to third-party financial systems. Read-only by default and failing closed, so an integration nobody configured cannot write
- Writes require three independent things: the integration declared writable, a token carrying a write scope, and an approval record for the specific change
Fast feedback
- `.budgets.exs` and `mix tally.budget`, enforcing time budgets for the test and quality runs
- `mix tally.budget.assets`, enforcing over-the-wire budgets on the built CSS and JS bundles
- CI base image carrying pre-compiled dependencies and a prebuilt dialyzer PLT, so CI compiles the application and nothing else
Documentation
- `AGENTS.md`, `docs/architecture.md`, `docs/design.md` and a dated decision log recording what was rejected and why
- **A failed model session does not end the call silently.** When Gemini Live closes on its own, for instance a quota refusal, `Tally.Voice.Live` publishes it, the trace records it, and the call ends rather than crashing. The Advisor page says the reason the model gave. The trace also records the model's token usage per call (from `usageMetadata`), because the Live API bills and throttles by token and a free-tier call can be ended by that limit.
- **Text into the live session.** The Advisor page has a field that joins the voice call: while a session is live, what is typed is streamed into the same Gemini session as a text turn, and the page renders it as its own bubble because the session's input transcription only covers spoken turns. Refused with a reason when no session is live, since a typed turn into a thread without a call belongs to the text agent's regime, not this one.