How to integrate Fish Speech API for automated long-form narration pipelines

Fish Speech is a game-changer for long-form content because it handles prosody way better than the old-school concatenation methods, but the API isn't exactly "plug-and-play" if you're trying to narrate a 20-minute script without it sounding like a robot having a stroke.

The biggest hurdle with long-form narration is the context window and the tendency for the model to drift in emotion or pitch over time. If you just dump a 5,000-word text into the API, you'll either hit a timeout or get a file that starts energetic and ends in a whisper.

My current pipeline solves this by implementing a "semantic chunking" layer before the API call. I use a simple regex-based splitter that respects sentence boundaries and paragraph breaks, ensuring no chunk exceeds 150 characters. This keeps the latency low and the intonation consistent.

Here is the core logic I use to handle the asynchronous requests and stitching:

import asyncio
import httpx

async def synthesize_chunk(client, text, reference_audio):
    payload = {
        "text": text,
        "reference_audio": reference_audio, # Base64 or URL
        "prompt_text": "The reference audio transcript",
        "format": "wav"
    }
    response = await client.post("https://api.fish.audio/v1/tts", json=payload)
    return response.content

async def process_long_script(script_chunks, ref_audio):
    async with httpx.AsyncClient(timeout=60.0) as client:
        tasks = [synthesize_chunk(client, chunk, ref_audio) for chunk in script_chunks]
        # Use gather to run concurrently, but watch your rate limits
        audio_segments = await asyncio.gather(*tasks)
        
    with open("final_narration.wav", "wb") as f:
        for segment in audio_segments:
            f.write(segment)

Pro Tips for Production:

The Reference Audio Trap: Don't use a 10-second clip of someone shouting for a calm narration. The model mimics the energy of the reference. I've found that a 3-5 second clip of neutral, clear speech works best for long-form. If you need a specific emotion, you have to swap the reference audio mid-script for different segments.

Avoiding the "Gap" Sound: When you stitch WAV files together, you sometimes get a tiny audible click or an unnatural silence. I use pydub to add a very short (50ms) crossfade or a precise silence buffer between chunks to make the transition seamless.

Rate Limit Management: If you're pushing hundreds of chunks, asyncio.gather will get you 429'd. I wrap my requests in a asyncio.Semaphore(5) to limit concurrent calls to 5 at a time.

The "Hallucination" Gotcha: Occasionally, Fish Speech will "hallucinate" a breath or a weird stutter if the text contains unusual punctuation. I've started scrubbing my scripts to remove double dashes (--) and replacing them with commas or periods to keep the synthesis stable.

The productivity gain here is massive. I went from spending 4 hours editing voice-overs in Audacity to spending 10 minutes refining a Python script and letting the API handle the heavy lifting. The key is treating the API as a series of small, controlled bursts rather than one giant request.