Why does maximizing Sliced Wasserstein Distance fail on

I've been obsessing over whether we can force two class distributions to push away from each other without completely wrecking the underlying geometry of the feature space. The logic seemed sound: if I can learn a transformation $T(x)$ that maximizes the Sliced Wasserstein Distance (SWD) between class A and class B, the resulting clusters should be more separable, making the downstream classification a breeze.

The problem is that if you just maximize SWD using a neural network, the model cheats. It finds the most degenerate transformation possible to blow the distance apart, which effectively destroys the actual relationships between data points. To stop this, I added some geometric constraints to the loss function to keep the transformation somewhat sane.

My pipeline looked like this:
Raw data → Learn transformation → Maximize SWD with geometric constraints → Standard ML classifier.

Here is the weird part. When I ran this on the Breast Cancer dataset, it actually worked for Decision Trees. But the moment I switched to almost any other algorithm, the performance tanked. It's like the transformation creates a specific kind of separation that only axis-aligned splits can exploit, while actually confusing models that rely on gradients or distance metrics like SVMs or Logistic Regression.

I'm starting to suspect that "maximizing distance" is a naive goal if you don't strictly define what "preserving geometry" means. If the transformation is non-linear, I might be creating these weird, thin filaments of data that a Decision Tree can isolate with a single cut, but which ruin the manifold for anything else.

For those who have done a deep dive into optimal transport or LLM agent feature engineering, does this sound like a regularization failure? I'm trying to figure out if there's a way to implement this as a practical tutorial for feature engineering or if I'm just chasing a ghost.

If anyone has a lead on a better constraint than basic geometric penalties, I'm all ears. I suspect the "solution" is actually just a distorted version of the data that happens to align with how DTs partition space.

# Conceptual loss function I was playing with
loss = -SWD(T(A), T(B)) + lambda * geometry_constraint(T(X), X)

Is this just a fancy way of doing a bad PCA, or is there a real-world use case for SWD-based separation that doesn't result in model-specific overfitting?