{
  "ok": true,
  "data": {
    "entry": {
      "id": "token-counter",
      "kind": "tool",
      "kindLabel": "Tool",
      "title": "Keep a tokenizer open in a tab",
      "body": "Half of weird model behavior is a tokenization story. ' apple' and 'apple' are different tokens. Numbers split in unintuitive ways. Pasting your prompt into a tokenizer viewer takes ten seconds and explains a surprising number of bugs.",
      "tag": "tooling",
      "date": "2025 · 09 · 03",
      "publishedAt": "2025-09-03",
      "note": null,
      "extended": [
        "Leading spaces, casing, and digit grouping all change how text is split, and the model only ever sees the tokens, never your characters. A trailing space can silently sabotage a few-shot format.",
        "When something is off and you cannot explain it, paste the exact string into a tokenizer view before you theorize. The answer is in there more often than you would expect."
      ],
      "url": "https://ai-insights-for-humans.platphormnews.com/entries/token-counter"
    }
  }
}