Same model, same prompt intent, different parsers. See why resilient parsing matters when LLMs wrap valid JSON in markdown fences or other formatting.
Quick Start
No comparison run yet
Select a preset above or paste your own JD and resume, then click Run Comparison.
When the 1,800-call benchmark ran (May 2026), the dominant failure mode for Claude 3 Haiku under naive Pydantic prompts was schema-echo: the model mirrored the $defs and properties JSON Schema wrapper from the prompt back into its response. Recent Haiku versions appear to have patched this behavior. This live demo therefore uses Gemini 2.0 Flash to reproduce a different failure mode that BAML's parser also handles: markdown code-fence wrapping. The underlying point is the same. LLM output reliability depends on a moving target of model-specific quirks, and BAML's value is handling those quirks consistently. The benchmark data in the eval repo represents the model snapshot at the time of the test.