Ei-Core Abstained From a Retention Forecast, and Its Threshold-Based Refusal Proved Correct

A VP of Customer Success posed a direct forecasting question during a client demo: "What's the 90-day retention forecast for the enterprise cohort that onboarded in Q1?"

Instead of generating a projection, the model declined:

I cannot provide a retention estimate for this cohort. 
Insufficient behavioral signal density (n=23 events) 
falls below the minimum threshold (n=150) for 
reliable projection. Recommend extending observation 
window or enriching with external firmographic data.

The refusal originated from a safety mechanism embedded during red-teaming six months ago — one that had slipped from active memory until it activated. The decision logic surfaced in the trace log:

# From the refusal trace
if effective_n < MIN_SAMPLE_THRESHOLD:
    raise AbstentionError(
        reason="insufficient_signal",
        metadata={"effective_n": effective_n, "threshold": MIN_SAMPLE_THRESHOLD}
    )

With only 23 engagement events recorded across 47 accounts, the data was too sparse to support a meaningful estimate. The model's choice to abstain prevented a hallucinated response from being presented as insight.

This behavior was not a bug but a designed feature — a calibrated threshold that overrides generative pressure when statistical reliability cannot be guaranteed. During the demo, the VP was shown the refusal directly, and the rationale was explained in context.

Two refinements remain under consideration:

1. The threshold value of 150 was derived from a synthetic benchmark, not production data. Calibration against real-world forecast error curves is needed.
2. Exposing internal constants like MIN_SAMPLE_THRESHOLD in user-facing output risks leaking implementation details. A standardized error code such as ERR_INSUFFICIENT_SIGNAL could replace it publicly, relegating specifics to backend logs.

Additionally, there is interest in building a recommendation layer that translates the missing feature set into a client-facing data-collection checklist — effectively turning the model's internal diagnostic into actionable guidance.

When a model's refusal aligns with statistical integrity rather than defaulting to generation, it shifts from failure to function. What approaches are teams using to design abstention UX in production systems?