One of the first steps in any modeling
project is to define the problem and establish the details of the
business objective. Many modelers want to grab some data and just
start building a model. Many Clients just want a model because
everyone else has one. The problem with this is that if the
parameters of the problem are not thoroughly discussed with the
business partners then the resultant model may end up with serious
flaws. Things to look for:
- Accuracy - The final model may not have sufficient accuracy to be useful.
- A model can have some performance accuracy but not enough to justify the expense of implementation and maintenance. It is important to understand what level of improvement is needed by the business customer.
- The wrong target and or performance window was not accurately defined.
- Estimate the potential accuracy of the model by researching similar models in the company or industry.
- Implementation - The model may not be implementable.
- Variables are used in the model that are very predictive but are not available for implementation.
- Variables are used in the model that are very predictive but are too expensive for implementation.
- Systems need to be revised or updated for real-time implementation.
- Evaluate the potential variables in the final model and check with the implementation team for known and unknown issues.
- Monitoring - The final model may not be adequately monitored and thus not be able to be subject to live validation.
- Monitoring is not best practice, it is a requirement in many industries.
- If a model cannot be monitored then it may not be known if and when it is no longer effective even hazardous to the business
No comments:
Post a Comment