Dr. Peter Quell is Head of the Portfolio Analytics Team for Market and Credit Risk in the Risk Controlling Unit of DZ BANK AG in Frankfurt. He is responsible for methodological aspects of Internal Risk Models and Economic Capital. He holds an MSc. in Mathematical Finance from Oxford University and a PhD in Mathematics. Peter is a member of the editorial board of the Journal of Risk Model Validation and a founding board member of the Model Risk Managers' International Association (mrmia.org).
Through this article, Quell highlights that the financial industry faces challenges regarding model risks associated with the use of machine learning techniques for risk management purposes.
Machine learning has become widespread in various fields where data-driven inferences are made. In the financial industry, its applications range from credit rating and loan approval processes for credit risk to automated trading, portfolio optimization, and scenario generation for market risk. Machine learning techniques can also be found in fraud prevention, anti-money laundering, efficiency, and cost control, as well as marketing models. These applications have shown significant benefits, and the financial industry continues to explore the use of machine learning.
However, the banking industry faces challenges regarding model risks associated with the use of machine learning techniques for risk management purposes. While regulatory guidance, such as the Fed's SR 11-7 and subsequent regulatory documents, provides comprehensive information, it may not address all the questions that financial practitioners have regarding the implementation and use of machine learning algorithms in their daily operations.
There is a clear need to share emerging best practices and develop a comprehensive framework to assess model risks in machine learning applications
One of the main challenges in applying machine learning in a regulatory context is explainability and interpretability. It is essential to be able to explain how the algorithm makes predictions or decisions for individual cases. Another challenge is overfitting, where algorithms perform well on training data but fail on unseen data. Robustness and adaptability are also crucial factors to consider, as markets and environments can change over time. Additionally, bias and adversarial attacks pose challenges unique to machine learning compared to classical statistics.
While some of these issues have been addressed within the machine learning community, it is crucial to transfer this knowledge to the banking industry without reinventing the wheel. The Model Risk Managers' International Association (mrmia.org) has issued a white paper discussing industry best practices in banking that can serve as a starting point, considering the rapidly evolving applications.
In response to these challenges, Model Risk Governance should also consider:
Model review: If machine learning algorithms frequently change their inner workings, how should model validation react? What should the validation activity cover, including aspects of conceptual soundness?
Model development, implementation, and use: How should the more prominent role of data be accounted for? What level of complexity can users handle? What kind of explanations would be accepted by users and senior management?
Model identification and registration: How should model complexity, the role of data, and model recalibration be accounted for in the model inventory?
Maintaining excellent quality standards: Existing frameworks need to be enhanced by additional checks for overfitting and sensitivity analysis to ensure robustness. Tests for possible bias and discrimination should also be reviewed to mitigate reputational risk.
While some banks have already developed frameworks to address model risks in machine learning applications, others are still exploring viable starting points. There is a clear need to share emerging best practices and develop a comprehensive framework to assess model risks in machine learning applications. Risk professionals are invited to share their views on model risk and machine learning with
aimrm@mrmia.org.
