Model Documentation Best Practices – To help Model Validators

With rise in regulatory scrutiny, modeling standards have increased and so their documentation. I have seen models which were developed 15 years back, they were not even documented!

As such no documentation is perfect, because model developer is a different person and validator is a different person. They both have different past experiences and hence different perspectives and hence different way of look at issues. The best documentations are those which help the validator to formulate the gaps in their understanding and make easier to ask right questions to the developers.

Still a lot more can be done by the model developers in improving their documentation. Some of them can be:

1.  Some models are model within model. Model developers thankfully are developing separate document for each model. They include flowcharts for the model where it is clearly demonstrated who are the feeder models and which ones are receiver models. It will be great if they also provide a flow chart of model documents as well. This flow chart will guide model validators where they should start reading so that they can understand the model without getting confused in minimum time.
   1. It is helpful if the model owners provide locations in the document (which will be many) about the various connecting models. These pointers will give location where conceptual details are provide.
2. In model documentation developers provide the thought process of the model development, statistical analysis which they have gone through. It is helpful and important. But when the model is model within model, it will be great if in each submodel documentation, model is described quickly in first few pages so that model validators are able to see the big picture quickly before going through complete documentation.
3. When there are multiple models in a model, the triangulation of the data is a big challenge. A document especially focusing this issue should be developed.
4. When model is within a model, a prototype model (favorably an excel based) should be developed which will help the model validators to understand the conceptual details of the model structure in an unambiguous manner.
   1. Clubbing this point with 2, it is important to have a quick mechanical overview of the model so that the validator has an understanding what are the components of the model, how are they mathematically related etc.
5. There are various excel sheets associated with the model. It will be really helpful if these excel sheets contain a tab which acts as a read me tab explaining the other tabs and inter-dependencies as well as what is the source of this excel sheet and where this excel sheet acts as an input.
   1. Developers often include various analysis to defend their model, they should document each of this analysis so that the validators may be able to replicate them. They should document each of the analysis’s purpose so that the validators are able to understand why this analysis adds value to the model’s defense.

This page will be updated regularly.