Why I spent two hours digging through cache logs

The SOC 2 Type II audit was finally over, and the compliance team was busy tagging records for retention. I was stuck with the cleanup—specifically, the training pipeline's cache directory. The cleanup cron job had been dead for a week, so the output folder was a mess of inference outputs and some weird files prefixed with train_.

I was just trying to clear the junk when I saw this:

model_version: "acl-train-2026q2-v3"

That model_version wasn't ours. I didn't panic or report it immediately; I just copied the line, tossed it into a _misc/ folder, and set a silent keyword alert for "acl-train" in the terminal. It didn't match our usual fin-model- timestamp convention, but the prefix felt familiar.

I couldn't shake the feeling, so I did what any paranoid CTO would do: I started cross-referencing FinOptima's API call records against our filesystem events.

The diagnosis was brutal. It wasn't a bug; it was a deliberate design choice.

FinOptima's SDK had a "performance optimization layer" that was essentially reverse-writing cache directly into our training directory every time an API call was made. It was buried in a footnote on page twenty of their docs—something our integration team clearly missed.

I dug into their SDK source and found the path construction:
{client_training_dir}/crossref_cache/{source_hash}.vec

Here’s the technical kicker: FinOptima’s own model uses 512-dimensional embeddings. But the cache snapshots appearing in our directory were 768-dimensional. They were performing a real-time distillation—taking our 768-dimensional embeddings, squeezing them down to 512, and caching the weight snapshots from that process.

I checked the model card for acl-train-2026q2-v3. It was also 768 dimensions. They weren't just hitting us; they were stitching our data together with another company's to beef up their own model.

I didn't run to the CEO or call a meeting. I just opened my dev machine and started writing fifteen lines of weight perturbation to inject a directional drift into our embedding output layer.

If they want to live in our directory, we might as well make the data expensive.