Join free

kits

The cost ledger

Records what every model call cost and how long it took, then shows you the twenty requests eating the budget.

Logs every model call to a local database and reports the latency distribution and the most expensive requests, so cost stops being a surprise on someone else's invoice.

Run it

  1. 1. Python 3.10 or newer. No dependencies at all: storage is sqlite from the standard library, so there is nothing to install and nothing to host.
  2. 2. Save this as ledger.py. import sqlite3, sys, time DB = "ledger.db" # Price per MILLION tokens, as (input, output). These change, and a stale price is # worse than no price, so the ledger refuses to guess: an unknown model raises. # Copy the current numbers from the provider's own pricing page. PRICES = { "example-model": (3.00, 15.00), } def _db(): c = sqlite3.connect(DB) c.execute("""CREATE TABLE IF NOT EXISTS calls( ts REAL, label TEXT, model TEXT, tin INT, tout INT, secs REAL, usd REAL)""") return c def record(model, tin, tout, secs, label=""): if model not in PRICES: raise KeyError(f"no price for {model!r}. Add it to PRICES from the provider's page.") pin, pout = PRICES[model] usd = (tin * pin + tout * pout) / 1_000_000 with _db() as c: c.execute("INSERT INTO calls VALUES (?,?,?,?,?,?,?)", (time.time(), label, model, tin, tout, secs, usd)) return usd class Call: """with Call("your-model", "summarise") as c: ... c.tokens(n_in, n_out)""" def __init__(self, model, label=""): self.model, self.label, self.tin, self.tout = model, label, 0, 0 def tokens(self, tin, tout): self.tin, self.tout = tin, tout def __enter__(self): self.t0 = time.perf_counter() return self def __exit__(self, *exc): record(self.model, self.tin, self.tout, time.perf_counter() - self.t0, self.label) def report(top=20): rows = _db().execute("SELECT label,model,tin,tout,secs,usd FROM calls").fetchall() if not rows: sys.exit("No calls recorded yet.") secs = sorted(r[4] for r in rows) total = sum(r[5] for r in rows) pct = lambda p: secs[min(int(len(secs) * p / 100), len(secs) - 1)] print(f"{len(rows)} calls total ${total:.2f} per 1000 requests ${total / len(rows) * 1000:.2f}") print(f"latency p50 {pct(50):.2f}s p95 {pct(95):.2f}s p99 {pct(99):.2f}s") print(f"\ntop {top} by cost. Read them, they usually share a shape:") for lbl, m, i, o, s, u in sorted(rows, key=lambda r: -r[5])[:top]: print(f" ${u:8.4f} {i:7d} in {o:6d} out {s:5.2f}s {m} {lbl}") if __name__ == "__main__": report()
  3. 3. Put the real prices in. Open PRICES and replace the example with the models you actually call, taking the numbers from the provider's own pricing page. An unknown model raises rather than costing zero, on purpose: a silent zero is how a cost report ends up confidently wrong.
  4. 4. Wrap your calls. Two lines around whatever you already have. from ledger import Call with Call("your-model", "summarise-ticket") as c: resp = your_existing_call(prompt) c.tokens(resp.usage.input_tokens, resp.usage.output_tokens) The label is free text. Use the step name, because that is how you will read the report later.
  5. 5. Run realistic traffic through it. Not five hand-picked prompts: the expensive requests are never the median, so a tidy sample hides exactly what you are looking for.
  6. 6. Read the report. python ledger.py
  7. 7. Change one thing, shorter context, a cache, a smaller model for the easy requests, and run it again. Keep both reports side by side.

What you get

One line of totals, a latency distribution at p50, p95 and p99, and the twenty most expensive calls with their labels. Read the twenty rather than the average, because they nearly always share a shape: an over-long context, a retry loop, a tool call that fires every time and helps rarely. That shape is your fix. The average is the number no user experiences, and p99 is the one who writes the review.

Take it further

  • Group by label to find which step in a chain carries the cost. It is rarely the one people assume.
  • Log the same rows to Langfuse or your own tracing tool as well, so cost sits beside the trace of the request that caused it.
  • Add a running daily total and a hard stop, so a runaway loop in a client environment exhausts a cap rather than a budget.
  • Multiply the per-thousand figure by expected volume before the first client conversation about price. Being wrong there is expensive in a way the code is not.