5‑Second Ad Review with Strict Pydantic Contracts

Spending 20–40 minutes on each ad creative just to verify brand rules? That’s how it felt before I built CreativeAudit, a hands‑on guide that squeezes that review into five seconds.

A plain chat with an LLM falls apart for this task: the output jumps formats, the model forgets constraints, you get free‑form text, and client data leaks to the cloud. For any real workflow that’s a deal‑breaker.

Design goals I forced myself to meet
1. Strict output contracts – the schema is a hard wall; if the LLM doesn’t obey it, the call fails, no fake scores appear.
2. Local inference – everything runs on‑premise with Ollama, so no data ever leaves the machine.
3. Prompts as code – Jinja2 files live in a versioned folder, making them editable without touching the app.
4. Binary compliance for any brand‑book violation; a 0 or 10 score removes subjectivity on the highest‑risk rule.
5. Human‑in‑the‑loop mode that first extracts structured data from messy input, shows it to the user for correction, then runs the evaluation.

Architecture at a glance

Brief + Creatives → Jinja2 Prompt → Local LLM (Ollama)  
↓  
Pydantic Validation  
↓  
Score + Verdict + Feedback

Key components:

• app/schemas.py – strict Pydantic v2 contracts that define the exact JSON shape.
• prompts/*.j2 – versioned prompt templates, treated like source code.
• app/main.py – orchestrates prompt assembly, LLM call, validation, scoring.
• demo/streamlit_app.py – thin UI layer for quick testing.

Engineering decisions that matter

• Pydantic as the contract – the model is inherently nondeterministic; treating the schema as absolute guarantees that malformed responses never masquerade as scores.
• Binary 0/10 for critical rules – a creative either fully complies with the brand book or it doesn’t; no “slightly violated” ambiguity.
• Jinja2 prompts – because they’re separate files, you can A/B test wording, review diffs, and keep the app code clean.
• Local Ollama inference – swapping between qwen2.5:7b, qwen2.5:14b or llama3.2 is a one‑line config change, keeping deployment lightweight.
• Smart input mode – managers often paste chat fragments; an initial LLM step normalizes that into structured JSON, which the user can edit before the final audit runs.

Scoring model
Each creative gets three numeric scores: brand_alignment (1‑10), constraint_compliance (0‑10), and message_clarity (1‑10). The total combines them with weights:

total = brand × 0.4 + compliance × 0.3 + clarity × 0.3

Verdicts are then derived from the total and any critical failures flagged by the binary check.

Testing strategy
I wrote 38 deterministic unit tests, mocking the external LLM so they run in under two seconds. Tests cover schema edge cases, malformed responses, connection hiccups, and the scoring logic, giving a reliable complete guide for anyone who wants to replicate the pipeline from scratch.

The whole system lives in a practical tutorial‑style repo, with clear step‑by‑step instructions for deployment, making it beginner‑friendly while still being a deep dive into real‑world LLM agent workflow for prompt engineering.

You are an ad compliance analyst. Given a creative description and a brand brief, output ONLY a JSON object that matches the schema below. Do not add any extra text.
{
  "verdict": "PASS|NEEDS_REVISION|FAIL",
  "brand_alignment": 1-10,
  "constraint_compliance": 0-10,
  "message_clarity": 1-10,
  "feedback": "short sentence explaining why"
}

CreativeAudit shows that a reliable LLM pipeline can replace hours of manual review with a fast, deterministic process that respects data privacy and enforces strict contracts. If you’re building any AI workflow that needs guaranteed output, this step‑by‑step setup is worth a look.