“Bank-grade” is one of the most used and least defined phrases in financial software. It is meant to reassure, and it usually does, which is exactly why it should be interrogated. Here is what we think the phrase should oblige a vendor to show, and how we try to meet that standard ourselves.
Four things the phrase should mean
Enforcement, not intention. A guarantee that lives in application code is a statement about how the team works. A guarantee enforced by the database is a property of the system. Ask a vendor where each rule is enforced; if the answer is “in our service layer”, the guarantee is only as strong as the next release.
Evidence, with sources. A number without a source is a slogan. “Thousands of tests” tells you nothing; “one hundred and twelve tests over a real database, counted on this date by this command” tells you something you can repeat. Every figure Ledger publishes carries the date it was re-counted and the command that counts it, and the site’s own build fails if a figure goes stale.
An adversary. Tests written by the team that wrote the code share its blind spots. Something else must try to break it: a separate audit function, working from the assumption that every claim is false until re-run live, whose refusals are recorded. Ledger’s first audit gate was refused. That refusal is published, because a history with no refusals in it is a history in which nobody looked hard.
Honesty about the gap. No system has done everything. The honest vendor says what was not exercised: which paths were not run, which figures were not measured, which certifications are not held. Ledger publishes no response-time figures because they were never measured on reference hardware; says plainly that no external body has audited it; and grades its own readiness for service with partials and gaps left visible.
What the phrase should not mean
It should not mean a certification badge alone. A certification says a set of controls was documented and sampled; it does not say the books cannot go out of balance. It should not mean uptime alone, either; a service that is always up and occasionally wrong is worse than one that is occasionally down. And it should never mean “trust us” dressed in a serious font.
How to check a vendor in an hour
- Ask for one guarantee, where it is enforced, and the test that guards it. Then ask to see the test fail on a broken input.
- Ask for the date and source of any number they quote.
- Ask who audited the system, whether that person could refuse a release, and whether they ever did.
- Ask what was not done. A vendor with nothing on that list has not made the list.
A worked example of a claim with a source
Consider the sentence “the ledger serves fifty-four live probes against the running service”. On its own it is a slogan. Here is what turns it into evidence:
- The count has a command. It is the number of probe scripts registered in the harness that runs against a served instance, and the exact command that produces the number is published beside it.
- The count has a date. It was taken on a stated day, and the site’s own build refuses to publish a figure older than four months.
- The probes have a discipline. Each proves it can detect a known defect before its clean result is trusted, and each tests its guard in both directions: that the guard blocks what it should and releases what it should.
- The probes have a history. Several were written because an audit round found a defect that the existing suite had missed. The suite grows with what it failed to catch, and the audit history says which rounds added what.
Any one of those four could be omitted and the sentence would still sound the same. That is exactly why the phrase “bank-grade” needs all four behind it.
Red flags, briefly
- A round number of tests, or a number with no date.
- “Audited” without saying by whom, whether they could refuse, and what they found.
- Guarantees described in the future tense, or with the word “designed to”.
- A response-time figure with no description of the hardware and load that produced it.
- A security page that lists controls but no way to verify a single one.
None of these prove a vendor is wrong. They prove the vendor has not shown you enough to know, and in a system that holds money, not knowing is the risk.
Ledger’s answers to all four are on the assurance and audit history pages. We would rather be checked than believed.