Why is performance inference important?

                                                        Return to FAMQ

Performance inference helps prevent the effects of “truncation.” If there is a serious chance that “truncation” is occurring, then performance inference should be included in the analysis.

In some cases, performance inference is less important than others. The primary importance depends on how the models are going to be used and how likely the inferred population is to be included in the scoring process. Also, how important is it to understand, for business knowledge the inferred population. For example:

• Credit Risk – If there is a sub group of applicants that will never be considered for credit (recent bankrupts, foreclosures, very low FICO scores …) then they might not need to be scored; however, if there is a reasonable chance that these credit policies might change then knowing how these policy rejects will perform is important for scoring purposes. It is very likely that the current rejects will perform worse than the “known” population.
• Insurance Risk – There may be an insurance policy that rejects any applicants who have had 5 accidents in the last year. As in credit, this policy might change and if it does it would be important to understand the insurance risk of this group to properly price them for insurance
• Competition – In both the risk examples above the are groups of applicants that walk away from an offer. This is often because they found a better offer elsewhere. In that case, it helps to understand the performance of these “walk aways” in order to make them offers that might be more competitive.
• Marketing – If a particular set of products is marketed to a very specific group and there is interest in expanding this market then it would be of interest to better understand the expanded population without going to the expense of actually marketing to that population.

The primary purpose of performance inference is to understand something about populations that the business is currently not serving. In the case of risk there is a lot of potential loss in “testing” into that population. In marketing or being more competitive, the risk is limited to the expense of marketing into a new arena or lowering prices to meet the competition. In both cases, performance inference will help in understanding the population with less expense and time.

                                                        Return to FAMQ