Introduction
JSON is well suited to APIs, storage and application logic, but inside large language model (LLM) pipelines it carries a lot of token overhead that adds little value for the model: braces, quotes, commas and repeated field names on every line. TOON — short for Token-Oriented Object Notation — is a format designed to keep the same JSON data model while using fewer tokens and giving the model clearer structural hints. Its documentation describes it as a compact, lossless representation of JSON for LLM input that is particularly effective on homogeneous arrays of objects.
This article explains what TOON is, when it is appropriate, and how to start using it in an LLM workflow, while keeping the trade-offs honest — TOON helps in some situations, not all.
Why JSON wastes tokens in LLM pipelines
JSON becomes expensive in prompts because it repeats structure over and over. A model does not care that JSON is a standard; it only sees tokens. When 100 support tickets, product lines or user records are sent to a model, the same field names appear in every object. TOON reduces that repetition by declaring fields once and then streaming the row values in a concise tabular form. A simple comparison illustrates the point.
JSON:
{
"users": (
{ "id": 1, "name": "Alice", "role": "admin" },
{ "id": 2, "name": "Bob", "role": "user" },
{ "id": 3, "name": "Charlie", "role": "user" }
)
}TOON:
users(3){id,name,role}:
1,Alice,admin
2,Bob,user
3,Charlie,userThe data is identical but the clutter is gone. The structure remains clear while the repeated keys disappear, which is where most of TOON’s savings come from.
What exactly is TOON, and when is it worth using?
TOON is a serialization format for the JSON data model, so it can represent objects, arrays, strings, numbers, booleans and null values — but in a form that is more compact for model inputs. The project presents it as lossless relative to JSON, meaning data can be converted from JSON to TOON and back without loss. Importantly, the application’s own JSON does not need to change; TOON is applied at the point where structured data is placed into a prompt.
The format borrows ideas from YAML for nested structure and from CSV for uniform arrays, replacing punctuation such as brackets and commas with indentation and line breaks, dropping unnecessary quotes, and declaring keys once in a tabular header. It tends to help most with large, uniformly structured collections — support tickets, analytics records, tool outputs, CRM entries or memory snapshots for an agent. Where data is deeply nested, highly irregular, completely flat or very small, the benefits shrink or disappear, and JSON, plain CSV or YAML may serve better.
Getting started with TOON
Step 1: Installing the TOON command-line interface
The easiest way to try TOON is through the project’s official command-line interface, which is linked from the TOON site and offered as part of a broader SDK and tooling ecosystem.
npm install -g @toon-format/cliStep 2: Converting a JSON file to TOON
The next step is to create a working folder, add a JSON file, and convert it.
mkdir toon-test
cd toon-test(
{ "id": 1, "name": "Alice", "role": "admin" },
{ "id": 2, "name": "Bob", "role": "user" },
{ "id": 3, "name": "Charlie", "role": "user" }
)npx @toon-format/cli users.json -o users.toonThe conversion produces a summary similar to the following:
(3){id,name,role}:
1,Alice,admin
2,Bob,user
3,Charlie,userThis is the core TOON pattern: declare the shape once, then list the values row by row, consistent with the format’s design goal of tabular arrays for uniformly structured data.
Step 3: Using TOON as model input
Once the data is in TOON, it is placed into the prompt wherever structured input would normally appear, so the model receives the same information encoded in various structured input formats with fewer tokens.
The following data is in TOON format.
users(3){id,name,role}:
1,Alice,admin
2,Bob,user
3,Charlie,user
Summarize the user roles and point out anything unusual.Step 4: Keeping JSON for output
This is one of the more important practical decisions. TOON is valuable for input, but JSON remains the better choice for output when another system needs to parse the model’s response, because JSON has strong tooling support and modern APIs can enforce structured JSON output against a schema. A safe pattern is therefore to keep JSON inside the application, use TOON for large structured input passed into the prompt, and return to JSON for machine-parsable model responses — gaining efficiency on the input side and reliability on the output side.
Step 5: Benchmarking the pipeline
Rather than switching formats on the strength of promotional claims, it is worth running a small benchmark within the actual workflow: counting input tokens for JSON and for TOON, then comparing latency, answer quality and total cost. The official project cites token savings as a primary benefit and third-party coverage echoes those claims, but community discussion notes that results depend heavily on the size and shape of the data. The useful question is therefore not whether TOON is better than JSON in general, but whether it is better for a particular stage of a particular pipeline.
Final thoughts
TOON is not a format to use everywhere. It is an optimisation aimed at a specific problem — wasting tokens on repeated JSON structure inside prompts. Pipelines that pass many uniformly structured records into a model are good candidates for testing it, whereas small, irregular or heavily nested payloads are often better left as JSON. The pragmatic approach is to keep JSON where it already works well, apply TOON where large structured inputs are packed into prompts, and benchmark the results on a real workload.
Limitations and what to watch
Reported token savings for TOON vary widely — independent write-ups cite figures spanning roughly 30% to 60% depending on the dataset — so any single percentage should be treated as conditional rather than guaranteed. The largest gains appear on big, homogeneous arrays of objects; on nested, irregular, flat or small data the savings can vanish, and a format such as CSV may be more compact still. Because TOON is a young format, its tooling, specification and ecosystem are still maturing, and support across libraries and platforms is far less universal than JSON’s.
There are also operational caveats. Fewer structural tokens can occasionally make data harder for a model to parse if the input is ambiguous, so accuracy should be validated rather than assumed, and output usually still belongs in JSON for reliable downstream parsing. As with any efficiency claim, the only dependable evidence comes from benchmarking on the specific model, data and task at hand. Background on the format is available from InfoQ and a comparative benchmark is published on arXiv. Readers building token-efficient retrieval may also find the walkthrough on building vector search in Python useful.
