Optimizing vLLM Throughput with PagedAttention and Custom LoRA Adapter Serving

Running vLLM in production often hits a wall when you try to scale multi-tenant LoRA adapters. If you're just loading full models, you're wasting VRAM; if you're switching adapters naively, your latency spikes. The magic happens when you leverage PagedAttention to manage the KV cache and specifically configure the LoRA adapter slots to avoid constant swapping.

The core issue is that while PagedAttention solves the fragmentation of the KV cache, the weights for your LoRA adapters still need to be managed efficiently. If you have 50 different fine-tuned adapters for different users, you can't keep them all in active memory without sacrificing batch size.

To get the highest throughput, I've found that setting the max_loras and max_lora_rank parameters during engine initialization is non-negotiable. If you leave these at defaults, vLLM might not allocate enough space in the GPU for the adapter weights, leading to frequent reloading from CPU RAM.

Here is the setup I use for a high-throughput adapter server:

python -m vllm.entrypoints.openai.api_server \
    --model /path/to/base_model \
    --enable-loras \
    --max-loras 16 \
    --max-lora-rank 64 \
    --gpu-memory-utilization 0.9 \
    --max-model-len 4096

Key tuning insights for this config:

• max-loras: This is the number of unique adapters that can be resident in GPU memory simultaneously. Set this based on your most frequent concurrent requests. If you have 100 adapters but only 10 are "hot," set this to 10-15.
• max-lora-rank: vLLM allocates memory based on the maximum rank you specify. If your adapters are all rank 8, don't set this to 64, or you're wasting precious VRAM that could have gone to the KV cache (increasing your max batch size).
• gpu-memory-utilization: When using LoRAs, I usually bump this to 0.9. PagedAttention is aggressive, but the adapter overhead is relatively small compared to the base model.

When actually calling the API, the throughput gain comes from the way vLLM batches requests. It doesn't run one adapter at a time; it uses a specialized kernel that can apply different LoRA weights to different tokens in the same batch.

To test if your throughput is actually scaling, stop looking at "tokens per second" for a single request and start looking at "aggregate tokens per second" across 10+ concurrent users using different adapters.

# Example of how to route requests to specific adapters via the OpenAI-compatible API
import requests

payload = {
    "model": "adapter_user_a", # This must match the name used during loading
    "messages": [{"role": "user", "content": "Analyze this log file..."}],
    "temperature": 0
}
response = requests.post("http://localhost:8000/v1/chat/completions", json=payload)

One gotcha I ran into: if you update an adapter on disk, vLLM doesn't automatically hot-reload it. You have to use the specific adapter management endpoints to unload and reload the weight files, or you'll be wondering why your "optimized" model is still giving old answers.

The biggest productivity gain here is moving away from "one model per container" to "one base model + N adapters." It reduces the VRAM footprint per user from 20GB+ down to a few hundred MBs, allowing you to cram significantly more throughput onto a single A100 or H100.