Challenges in Vendor Model Validation in Financial Risk Management

1)  In vendor model documentation, model flow diagram is provided. These flow diagrams explain the conceptual framework but a more detailed model architecture, mapping outlining all model components including specific flow/transformation of data from input to final output should be provided.

2) Many models use multiple inputs from internal and external sources. There should be transparent evidence of their validity and authenticity.

3) A detailed document should be provided for calibration of pricing and risk models. An importance case is Value at Risk or Potential Future calculation using Monte Carlo simulation. There are various correlation assumption made in aggregation of the portfolio returns and scenario generation. These correlation assumptions should be clearly documented.

4) Validator should perform exhaustive sensitivity analysis. They should study the model output by changing each input variables and thus study variable’s linear or non linear relationship with the output.

5) When the validator is validating a vendor model which is replacing another vendor model, multiple subsets of input data should be created covering maximum possible business cases and output values of the new model should be benchmarked with the output values of the older model.

6) Validator should have clear understanding on how model treats the missing values. If documentation is not available then by running the model multiple times and thus make a judgment.

7) Often there will be times when vendor model output wont be intuitive to model owner or business owner. That will not mean that the model is conceptually wrong. Deep dive analysis by model or business owner will not be be possible because model methodology being proprietary. To manage model risk a process plan should be in place where detailed issue can be passed to the vendor to resolve the issue in efficient and minimum time and iteration.

 

For detailed discussion on Vendor Model Validation please refer to the below white paper from PSS Moorthy:

Vendor Model Validation: Challenges and Proposed Solutions

http://www.garp.org/#!/risk_intelligence_detail/a1Z40000000HyInEAK

This post will be updated in future.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s