I deployed a microservices setup for a media extractor and it

Building a tool that needs to scrape and process data from over 1,000 different platforms like TikTok, YouTube, and Instagram is a nightmare if you try to do it in a single monolithic block. I recently finished FastMedia Downloader, and the biggest lesson I learned wasn't about the scraping itself, but about how to isolate heavy-duty dependencies using a microservices approach.

If you've ever tried to install FFmpeg or yt-dlp directly on your host machine only to have a version conflict break your entire development environment, you'll understand why I went the Docker route immediately.

The Architecture Breakdown

I split the entire system into two distinct, containerized services. This keeps the frontend logic completely decoupled from the heavy lifting happening in the backend.

• Backend Service: Built with FastAPI and Python 3.11. This acts as the orchestrator. It handles all the asynchronous requests, media inspection, and the actual conversion logic. I'm using custom wrappers around yt-dlp and FFmpeg to manage the heavy media streams.
• Frontend Service: A Next.js application using TypeScript and Tailwind CSS. It’s designed to be a lightweight interface that handles instant link parsing and provides multi-language support (i18n).
• Orchestration: I used Docker Compose to manage the lifecycle of both services. This makes the entire deployment a single command: docker-compose up.

Implementing the Async Workflow

One of the most critical parts of this project was managing the latency during media inspection. When a user pastes a link, you don't want the entire API to hang while the backend reaches out to a third-party server.

I focused heavily on async handling within FastAPI. By treating the metadata extraction as an asynchronous task, the backend can keep response times low even when dealing with slow external APIs.

Containerizing System Dependencies

The real "pro tip" here is how I handled the system-level binaries. Instead of assuming the user has the right version of FFmpeg installed, I encapsulated those binaries directly inside the backend Docker container.

Here is a simplified look at how the Docker setup manages that environment:

# Example snippet of the backend environment setup
FROM python:3.11-slim

# Install system dependencies directly in the container
RUN apt-get update && apt-get install -y \
    ffmpeg \
    yt-dlp \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

This approach ensures that "it works on my machine" actually means it works on every machine. Whether you are running this on a local dev machine or deploying to a cloud provider, the environment remains identical.

Deployment and CI/CD

I also integrated a standard DevOps workflow to keep the repo clean. I'm using GitHub Actions for CI/CD and Dependabot to monitor vulnerabilities. If you are looking for a practical tutorial on how to structure a production-ready open-source repo, I'd recommend looking at the repository structure I used.

The full source code is available here if you want to fork it or dive into the specific implementation of the yt-dlp wrappers:

https://github.com/Llamas126/fastmedia-downloader