top of page

Deep Dive: Relevance

Friday, November 6, 2026

Relevance is the cornerstone of relevance-based prediction (RBP). Without relevance, it is impossible to compute fit, which measures the conviction one should assign to a prediction. And without fit, it is impossible to carry out grid prediction which produces a reliability-weighted composite prediction for an outcome. And without grid prediction, it is impossible to measure a predictive variable’s importance to the reliability of a prediction. Therefore, to grasp the power and versatility of RBP it is essential to understand relevance: its intuition, its theoretical justification, its connection to important statistical concepts, its calculation, and how it is used to form a prediction. 


Intuition: Relevance is a statistic that gives a precise measure of an observation’s importance to a prediction. Although the formula that defines relevance is complicated, its intuition is simple because it matches the way we naturally contemplate the future. When faced with a set of circumstances from which we would like to consider the future, we naturally recall past experiences that are like current circumstances, and we recollect what followed from those past experiences. Moreover, we tend to have a better recollection of past experiences that were unusual rather than common experiences. Relevance aligns with our natural approach to contemplating the future. It has two components: similarity and informativeness. Informativeness is measured as unusualness, because unusual observations contain more information than common observations. (More about this later.) Therefore, observations that are like current circumstances but different from average circumstances are more relevant than those that are not and, thus, more important to forming a prediction.


Theoretical justification: It is important to appreciate that relevance is not arbitrary. It is supported by the Central Limit Theorem, which holds that the sum or average of independent random variables, that individually may not be normally distributed, approaches a normal distribution as the number of random variables increases. Relevance is also supported by information theory, which holds that information is an inverse logarithmic function of probability, which is to say unusual observations contain more information than common observations. You may recall from your introductory statistics class that you were warned to be skeptical of outliers. Information theory belies this view about outliers, assuming they are not data errors. Even if an outlier represents an outcome that may never recur, we should view it as a natural experiment in which the true relationship between a predictive variable and an outcome emerges but would otherwise be hidden if the values only fluctuated within a narrow range. 


Connection to important statistical concepts: RBP’s connection to important statistical concepts gives additional evidence that relevance is not arbitrary. RBP forms a prediction as a relevance-weighted average of observed outcomes for a given sample of observations. If RBP is applied to the full sample of observations, it gives precisely the same prediction as linear regression analysis. It, therefore, enjoys the same theoretical justification that Carl Gauss gave for linear regression analysis circa 1795. Additionally, fit which measures the alignment of relevance and outcomes between pairs of observations, and which gives a prediction-specific measure of a prediction’s reliability, converges precisely to R-squared when it is averaged across all prediction tasks. Finally, the breakthrough that enabled ChatGPT and other large language models (LLMs) is relevance. LLMs use attention to predict the contextualized meanings of words from their unconditional meanings, and attention is essentially relevance.


Calculation: The relevance of an observation to a prediction is calculated as the sum of similarity and informativeness, which are both measured as Mahalanobis distances. Similarity captures how close the values of a set of predictive variables for a given observation are to the values of those predictive variables for the prediction task at hand, considering not just the variables independently, but their interactions as well. Informativeness captures how different the values of a set of predictive variables for a given observation are to the average values of those predictive variables, again considering not just the variables independently, but also their interactions. Moreover, relevance includes not only the informativeness of the observation under consideration but the informativeness of the prediction task at hand, which conveniently centers relevance on zero across all observations. The Mahalanobis distance, which measures similarity and informativeness, is an elegant formula that was first derived in 1936 by an Indian mathematician named Prasanta Mahalanobis to analyze human skulls as part of a study about mixed parentage between the British and different Indian castes. It has since been used to measure financial turbulence, to diagnose diseases, and to detect anomalies in autonomous vehicles, among other applications. It has two powerful features. It accounts for variation across the values of the variables it is considering, and it accounts for their interactions. Because it is constructed to measure difference, multiplying it by -1 gives a measure of similarity.


How relevance is used to form a prediction: RBP forms a prediction as a weighted average of observed outcomes in which the weights are based on relevance. Its equivalence with linear regression analysis when applied to the full sample of observations reveals an interesting insight. Linear regression analysis places as much importance on non-relevant observations as it does on relevant ones; it just changes the sign of how the non-relevant observations inform the prediction. RBP presumes it is possible, if not likely, that we can form more reliable predictions by using a subsample of relevant observations. RBP also presumes that we should tailor the choice of the relevant subsample to the specific prediction circumstances and that we should select the predictive variables in light of the choice of observations and the prediction circumstances. RBP, therefore, forms many predictions from different combinations of observations and predictive variables, and forms a composite prediction as a weighted average of all the predictions in which the weights are the relative reliabilities of the predictions, as determined by the strength of the patterns in the observations associated with the different combinations.   It is important to note that RBP is not a model with fixed parameters. It is a prediction routine that retrieves the original observations for each new prediction task, which renders the formation of each prediction fully transparent, an important advantage of RBP compared to model-based prediction techniques.

©2026 Cambridge Sports Analytics, a division of Cambridge Prediction Analytics, LLC. All rights reserved.      

Cambridge, Massachusetts

bottom of page