Laravel AI Ops: Handling Queues and Timeouts

LLM calls are absolute nightmares for standard server configurations because they violate every "fast request" assumption we've built our stacks on. Most of our servers are tuned for 200ms responses, but an LLM might take 60 seconds to think, and if you're not careful, your Nginx config or PHP-FPM settings will just kill the connection while the model is mid-sentence.

As a game dev, I'm obsessed with performance, and nothing kills the UX faster than a 504 Gateway Timeout because the server gave up on a prompt. Here is the breakdown of why your defaults are lying to you and how to actually fix it for a real-world AI workflow.

The "Timeout" Minefield

If you're running a standard VPS, you're probably hitting these walls:

PHP max_execution_time: Default is usually 30s. A complex prompt that takes 45s? Dead.

Nginx fastcgi_read_timeout: Usually 60s. If the model hangs, Nginx returns a 504 while the LLM is actually still processing.

FastCGI Buffering: This is the silent killer for streaming. Tokens get stuck in the buffer, and the user sees a blank screen for 10 seconds followed by a giant wall of text.

Queue retry_after: If this is shorter than your LLM response time, a second worker will pick up the "timed out" job, and you'll pay for the same prompt twice.

The Golden Rule: Offload Everything

Stop making synchronous LLM calls in your controllers. Tying up a PHP-FPM worker for a minute is a great way to crash your entire site when five people use your AI feature at once. Move everything to queued jobs.

I've put together a practical tutorial for a job skeleton that handles the common pitfalls. This is how you should be structuring your LLM agents to ensure they don't blow up your server or your budget.

namespace App\Jobs;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Http;
class ProcessAiPrompt implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
    // Give the job plenty of time to finish before the queue thinks it failed
    public $timeout = 300; 
    public function handle(): void
    {
        // Use a generous timeout for the HTTP client specifically
        $response = Http::timeout(120)->post('https://api.openai.com/v1/chat/completions', [
            'model' => 'gpt-4o',
            'messages' => [
                ['role' => 'user', 'content' => $this->prompt],
            ],
        ]);
        if ($response->failed()) {
            // Handle retry logic based on status code to save money
            $this->handleFailure($response);
        }
    }
}

For a complete guide on fine-tuning your environment, I recommend checking out the deployment docs at promptcube3.com. Focus on increasing your retry_after setting in config/queue.php to be significantly higher than your longest expected LLM response time—otherwise, you're just burning credits.