Gemini Vision can actually tell if your dog is judging you

I decided to build a small app that uses Gemini Vision to analyze dog expressions and return a "judgment percentage." It's a goofy idea, but the technical implementation was a great way to test out some specific AI workflow patterns, especially regarding structured data and deployment.

The core of the app is a SvelteKit 5 frontend (using the new runes API) and Tailwind CSS 4, hosted on Netlify. The main goal wasn't just to make a prompt that works, but to build a production-ready tool that doesn't crash or leak user data.

The technical breakdown

The most important part of the AI workflow here is how the image is handled before it even hits the LLM. I implemented a client-side pipeline where the photo is drawn to a canvas and re-exported as a compressed JPEG. This serves two purposes: it keeps the payload small for faster API responses and automatically strips EXIF metadata (like GPS coordinates), which is a must for any real-world deployment.

For the backend, I'm using a SvelteKit API route that hits gemini-2.5-flash. The "secret sauce" for reliability here is the responseSchema. Instead of the usual "please return JSON" prompt—which often fails or includes annoying markdown backticks—the schema forces the model to adhere to a strict structure.

Here is the basic logic for the API call:

// Simplified logic for the /api/judge route
const result = await model.generateContent({
  contents: [{ role: 'user', parts: [imagePart, textPart] }],
  generationConfig: {
    responseMimeType: 'application/json',
    responseSchema: {
      type: 'object',
      properties: {
        judgmentLevel: { type: 'number' },
        emotion: { type: 'string' },
        innerMonologue: { type: 'string' },
        advice: { type: 'string' },
        breedGuess: { type: 'string' },
      },
      required: ['judgmentLevel', 'emotion', 'innerMonologue', 'advice', 'breedGuess'],
    },
  },
});

Handling scale and stability

Since this is a public-facing app, I couldn't just leave the API open. I integrated Upstash Redis to handle rate limiting using a sliding-window approach (capped at 5 requests per hour per IP). This is way more reliable than in-memory limiting because it persists across serverless cold starts.

I also added several "boring" but critical features to move this from a prototype to a real app:

• Input Validation: Strict checks on file size and MIME types.
• Timeouts: The Gemini call has a hard timeout to prevent hanging requests.
• Error States: Proper frontend feedback if the AI service is down or the API key hits a limit.

The final polish includes canvas-confetti that triggers based on the judgment score and a dynamic prompt that uses navigator.language to generate the dog's "inner monologue" in the user's native tongue. It's a simple project, but it proves how powerful combining a vision model with a strict schema can be for building lightweight, interactive AI tools.