Gemini Business Card Merge: My AI Workflow

Handling double-sided business cards is a nightmare when your bot is too "stupid" to realize that the English side of a card isn't a different person. My previous setup was a disaster: user sends the front (Chinese), bot saves it. User sends the back (English), bot saves a second record. Congratulations, you now have two half-empty entries for the same guy.

Instead of writing a thousand fragile regex rules to guess if two names are the same, I just let the user tell me. I implemented a "waiting room" state. When the first side hits, the bot asks, "Is there a back side?" If the user says yes, the bot holds onto that image in a user_states dictionary with a 5-minute timeout (because users have the attention span of a goldfish).

The real magic happens when both images are fed into Gemini simultaneously. Trying to merge "Wang Daming" and "David Wang" via code is a recipe for a headache. Sending both images in one prompt and letting the LLM handle the semantic merge is the only sane way to do this.

Here is the prompt logic I used to force Gemini to combine the data into a single clean JSON object:

You are an expert OCR and data extraction agent. I am providing you with two images: the front and back of a single business card. 

Your task:
1. Extract all professional information from both images.
2. Merge the data into a single unified record.
3. If a field (like Name or Company) appears in both languages, combine them into a single string (e.g., "Chinese Name / English Name").
4. Ensure the output strictly follows the provided JSON schema.
5. Ignore any background noise or irrelevant text.

Return only the final merged JSON.

For the implementation, I used gemini-1.5-flash because it's fast enough that the user doesn't forget why they're waiting. I just wrap both image parts into the generate_content call.

def generate_json_from_two_images(
 front_img: PIL.Image.Image,
 back_img: PIL.Image.Image,
 prompt: str) -> object:
 model = GenerativeModel(
 "gemini-1.5-flash",
 generation_config={
 "response_mime_type": "application/json",
 "response_schema": NAMECARD_SCHEMA
 },
 )
 front_part = Part.from_data(
 data=pil_to_bytes(front_img), mime_type="image/jpeg")
 back_part = Part.from_data(
 data=pil_to_bytes(back_img), mime_type="image/jpeg")
 response = model.generate_content(
 [prompt, front_part, back_part],
 stream=False,
 labels={"client_id": "namecard"}
 )
 return response

This approach turns a messy data-cleaning chore into a simple prompt engineering task. No more duplicate entries, no more manual deleting—just a clean, consolidated record.