Modeling Through Change
How We Keep Our Models Reliable Through Industry and Institutional Changes
At Liaison Othot, we understand that higher education industry standards, regulations, and data management practices can differ from year to year. While some changes are external and beyond an institution's control, others may be internal, such as updates to systems, shifting data elements, or evolving data collection methods.
Othot’s models are built on the relevance and consistency of historical data. However, we recognize that change is inevitable.
When substantial changes occur, we take a comprehensive and adaptive approach to ensure our models remain robust and relevant. The process is structured as follows:
Client-by-Client Model Review
We review each client’s model instance and data environment on an annual basis when incorporating the latest year’s outcomes, ensuring models stay aligned with current realities. At the same time, we remain flexible and can address both external and internal changes as they arise. Because the impact of these changes can vary by institution, we assess each situation individually and tailor our approach to meet the unique needs of every client.
Historical Data Selection and Model Training
Our standard practice is to build models using the two most recent years of historical data. However, if significant changes affect the comparability or utility of past data—or if performance is impacted—we may adjust the data window (e.g., using years that best align with newly available trends and elements for the cycle ahead).
Adapting to Changes in Data Elements
The approach to adaptation depends on the changes we see.
- Removed Data Elements: If certain data points will no longer be available, we remove them from all impacted models for future cycles.
- Added Data Elements: Newly introduced fields are incorporated only when sufficient representative data has been collected.
- Modified Data Elements: If variables change in definition or calculation, we attempt to map or transform historical values to match the new format. If this is not feasible, those elements are removed until adequate new data accrues, though they remain accessible for analysis outside core model predictions.
Additional or Interim Models
If industry or institutional changes result in delayed or staggered data availability, we may introduce interim models to reflect shifting timelines or information gaps. These models are designed to maintain prediction accuracy even when critical new data is pending.
Proactive Change Monitoring
We work to identify, understand, and anticipate industry changes before they impact institutional operations or predictions. This helps all institutions prepare and adapt well ahead of upcoming cycles.
Institution-Specific Adjustments
By maintaining a flexible, responsive modeling approach, we ensure our clients benefit from models that are up-to-date, fair, and effective, even as circumstances evolve. Our goal is always to find the optimal balance between leveraging historical trends and adapting to new, sometimes rapidly emerging, realities. If you have questions about how a specific change might affect your models, please reach out to your client success director or contact us.