My Rust hot path just went from 1184 ns/op to 43 ns/op

I spent the last few weeks rebuilding the audio engine for KeyEcho, a mechanical keyboard sound simulator I open-sourced a while back. The goal wasn't just "optimization"—it was a complete architectural overhaul. I had to move from a model that was "fast enough" to one that leaves zero headroom for latency. My microbenchmarks showed that after the rewrite, I went from copying 66.84 KiB of data per keypress to zero bytes copied. That's a 27x speedup on average slices and 38x on the largest ones.

The bottleneck in v0.0.5 was obvious once I looked at the telemetry: even with an LRU cache, every single keypress was hitting a global mutex used for volume changes and pack switching. It was also performing sample copies every time a sound played. In a latency-sensitive path like a keyboard hook, every allocation and every lock is a tax you pay on every single keystroke.

The 1.0 rebuild required a fundamental shift in how I handled the audio data. I implemented a pre-decode strategy. Instead of decoding on a cache miss, the system now decodes the entire sound pack into memory when a pack is selected.

To keep the memory footprint sane, I used a deduplication pattern. Since many keys in a sound pack map to the same audio slice, I used a HashMap to ensure each unique slice is decoded exactly once.

// Decode each unique slice once; identical slices share one buffer.
let mut slice_sources: HashMap<[u32; 2], AudioSource> = HashMap::new();
let mut key_sources: HashMap<Key, AudioSource> = HashMap::new();
for (key, slice) in defines {
 let source = slice_sources
 .entry(slice)
 .or_insert_with(|| {
 let [start_ms, duration_ms] = slice;
 let samples = decode(start_ms, duration_ms); // Vec, decoded once
 AudioSource::new(Arc::from(samples), channels, sample_rate)
 })
 .clone(); // This is an Arc clone, not a sample copy
 key_sources.insert(key, source);
}

The real magic is the zero-copy playback. By wrapping the decoded samples in an Arc, the per-key lookup is just a reference-count bump. More importantly, I had to kill the global mutex. I replaced the shared state with ArcSwapOption for the soundpack and AtomicU32 for the volume (using f32::to_bits). Now, the playback handle performs a lock-free load.

struct PlaybackSoundpack {
 current_sound: ArcSwapOption<Arc<SoundData>>,
 volume_bits: Arc<AtomicU32>,
}
fn source_for_key(&self, key: Key) -> Option<Arc<SoundData>> {
 let sound = self.current_sound.load();
 // Lock-free access pattern
 sound.get_slice(key)
}

I actually used an AI agent to help scaffold some of this heavy lifting, which was productive until it tried to get "clever." It suggested a refactor that would have moved the sound lookup into a separate thread pool to "reduce jitter," but it would have introduced massive context-switching overhead for such a short path. I refused to merge it. In low-latency systems, sometimes the simplest, most direct path is the only one that works.