Effective Few-Shot Prompting Techniques for Improving JSON Output Consistency

DeepSeek-V3 and GPT-4o handle JSON schema adherence quite differently when you push them with complex, nested structures. If you're relying on "JSON mode" alone, you'll eventually hit a wall where the model hallucinates keys or fails to escape special characters in string values. I've spent the last few days benchmarking few-shot patterns to see which actually stabilizes the output across different providers.

The biggest mistake most people make is providing a single, perfect example. In my tests, a single shot often leads the model to over-fit to that specific example's data patterns rather than the schema itself. To get consistent JSON, you need "contrastive examples"—one example that is simple and one that is an edge case (e.g., empty lists, null values, or extremely long strings).

Claude 3.5 Sonnet is currently the gold standard for following structural constraints. It doesn't need as many shots as Gemini 1.5 Pro, which tends to drift if the prompt is too long. For Claude, I found that placing the examples after the system instructions but before the user input works best.

Here is the prompt structure that cut my parsing error rate by about 15% on a dataset of 500 complex extractions:

System: You are a data extraction engine. Output strictly valid JSON.
Schema: { "user_id": int, "tags": list[str], "metadata": { "source": str, "confidence": float } }

Example 1 (Standard):
Input: "User 123 logged in from Web."
Output: {"user_id": 123, "tags": [], "metadata": {"source": "Web", "confidence": 1.0}}

Example 2 (Edge Case):
Input: "Unknown user from an unidentified source."
Output: {"user_id": null, "tags": [], "metadata": {"source": "unknown", "confidence": 0.0}}

User: [Actual Input]

When comparing the performance:

GPT-4o
Pros: Extremely fast convergence; usually only needs one example to get the format right.
Cons: Tends to add conversational filler ("Here is the JSON:") unless you are very aggressive with the system prompt or use a strict API mode.

DeepSeek-V3
Pros: Surprisingly robust with complex nesting; handles technical jargon inside JSON values better than GPT-4o.
Cons: Occasionally ignores the null requirement and substitutes it with an empty string "" if the few-shot examples aren't explicit about nullability.

Claude 3.5 Sonnet
Pros: Highest adherence to schema; least likely to "hallucinate" new keys.
Cons: More sensitive to the order of examples; if the last example is too simple, it might over-simplify the actual output.

If you're seeing the model truncate the JSON or miss closing braces, stop adding more examples and start using a "delimited" approach. I've found that wrapping the examples in XML-style tags (e.g., <example>...</example>) helps Gemini and Claude distinguish between the pattern they should follow and the instructions they should obey.

For those working with DeepSeek, I recommend explicitly defining the type of each field in the prompt—not just in the JSON example, but as a bulleted list above the examples. It forces the model to treat the schema as a constraint rather than just a pattern to mimic.