(This post is yet to be updated …)
With the arrival of SR7-11 guidance model risk management is mandatory exercise for the risk managers. They not only have to defend their models in front of the senior management and business units but also need to defend them in front of the regulators.
In financial world, model risk is unexpected inability of the model to capture the risk for which the model was designed. This results in inappropriate capital allocation for the risk which was to be managed by the particular model.
As per SR7-11, model risk primarily occurs for two reasons:
1) The model may have fundamental errors and may produce inaccurate outputs when viewed against the decision objective and intended business uses.
2) The model may be used incorrectly or inappropriately.
Current regulatory guidelines demand that there should be two separate and independent teams to manage financial risk models. One model developers, their job is to co-ordinate with the business and design a model which captures the risk of the business and hence allocate right capital. Second model validators, their job is to validate the assumptions made by the model developers, validate the concepts they have used measure the objective risk, check if the model adequately quantifies the objective risk. The validators act as a last line of defense against any model risk. They are the ones who have to defend their validated models to the regulators.
The regulators require that risk managers should have a clear understanding of risk management exercise they are doing. Model owners were never ignorant about the limitations of their models in the past. They are the most close teams to the business owners. The job of model validators is pose questions to the model owners so that they may document that understanding and hence business owners may have a clear view about the limitations of the risk models and deviations these models make from the actual world.
Statistical and mathematical soundness of the model is important but truth is that what may be conceptually sound today may not be applicable tomorrow. Black Scholes option pricing theory is a classic example. Four decade back this concept arrived and changed the financial markets forever. They even received Nobel price in economics. But today rarely options are priced using this concept because it is well recognized fact that stock returns for non-normal distributions contrary to the assumption of Black Scholes.
So in order to manage model risk, the risk managers should first consider the use cases. They should ensure if the model output is appropriate to the intended purpose. Is the output properly bench marked, how valid are these bench marks. For bench marking risk managers should not rely on single sources. This is specially important for the valuation models.
Second thing which risk managers should consider is the implementation process. They should ensure:
1) Is the model documentation good enough to be replicated by a completely new set of team?
2) Most of the models are spread sheet models. Are these spread sheets properly linked? Is there explanation of each and every hard coded number?
3) Are the challenges about data collection methodology documented?
4) How sustainable is the process around the considered model?
Third thing which risk managers should consider is the monitoring process. They should ensure:
1) There logs maintained for every change in the modeling process, these logs should contain detailed reasons of those changes.
2) Documentation of discussion of all communication happened about the model with approvals of all the stake holders.
A model is actually an opinion of an analyst. This opinion is based on available data and resources. With change in data and resources the opinion will change, rather it should change. As discussed, Black Scholes is a classic example. The regulars understand the fact around the limitations of data and resources. They understand that a model is simplification of real world and by noway it can capture the real world processes. Within SR7-11 guidelines they expect the risk managers to have a clear understanding and knowledge where their assumptions are deviating from the real world. Having a detailed understanding and documentation of use cases, implementation process and model monitoring can give used desired depth in that understanding.