1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Statistics and Its Application
  4. Volume 9, 2022
  5. Article

Annual Review of Statistics and Its Application

Volume 9, 2022

Risk Measures: Robustness, Elicitability, and Backtesting

Abstract

Risk measures are used not only for financial institutions’ internal risk management but also for external regulation (e.g., in the Basel Accord for calculating the regulatory capital requirements for financial institutions). Though fundamental in risk management, how to select a good risk measure is a controversial issue. We review the literature on risk measures, particularly on issues such as subadditivity, robustness, elicitability, and backtesting. We also aim to clarify some misconceptions and confusions in the literature. In particular, we argue that, despite lacking some mathematical convenience, the median shortfall—that is, the median of the tail loss distribution—is a better option than the expected shortfall for setting the Basel Accords capital requirements due to statistical and economic considerations such as capturing tail risk, robustness, elicitability, backtesting, and surplus invariance.

Keyword(s): backtesting, elicitability, expected shortfall, median shortfall, risk measures, robustness, value-at-risk
Loading

Article metrics loading...

/content/journals/10.1146/annurev-statistics-030718-105122
2022-03-07
2024-05-03
Loading full text...

Full text loading...

/deliver/fulltext/statistics/9/1/annurev-statistics-030718-105122.html?itemId=/content/journals/10.1146/annurev-statistics-030718-105122&mimeType=html&fmt=ahah

Literature Cited

  1. Acerbi C. 2002. Spectral measures of risk: a coherent representation of subjective risk aversion. J. Bank. Finance 26:71505–18
     [Google Scholar]
  2. Acerbi C, Székely B. 2014. Back-testing expected shortfall. Risk Mag. 27:1176–81
     [Google Scholar]
  3. Adrian T, Shin HS 2014. Procyclical leverage and value-at-risk. Rev. Financ. Stud. 27:2373–403
     [Google Scholar]
  4. Artzner P, Delbaen F, Eber JM, Heath D. 1999. Coherent measures of risk. Math. Finance 9:3203–28
     [Google Scholar]
  5. Artzner P, Delbaen F, Koch-Medina P. 2009. Risk measures and efficient use of capital. Astin Bull. 39:1101–16
     [Google Scholar]
  6. Basel Comm. Bank. Superv 1996. Amendment to the capital accord to incorporate market risks. Work. Pap., Bank Int. Settl. Basel, Switz:.
  7. Basel Comm. Bank. Superv 2006. International convergence of capital measurement and capital standards: a revised framework. Work. Pap., Bank Int. Settl. Basel, Switz:.
  8. Basel Comm. Bank. Superv 2009. Revisions to the Basel II market risk framework. Work. Pap., Bank Int. Settl. Basel, Switz:.
  9. Basel Comm. Bank. Superv 2013. Fundamental review of the trading book: a revised market risk framework. Work. Pap., Bank Int. Settl. Basel, Switz:.
  10. Basel Comm. Bank. Superv 2019. Minimal capital requirement for market risk. Tech. Rep., Bank Int. Settl. Basel, Switz:.
  11. Bellini F, Bignozzi V. 2015. On elicitable risk measures. Quant. Finance 15:5725–33
     [Google Scholar]
  12. Berkowitz J. 2001. Testing density forecasts, applications to risk management. J. Bus. Econ. Stat. 19:465–74
     [Google Scholar]
  13. Berkowitz J, Christoffersen P, Pelletier D 2011. Evaluating value-at-risk models with desk-level data. Manag. Sci. 57:2213–27
     [Google Scholar]
  14. Brunnermeier MK, Crockett A, Goodhart C, Persaud AD, Shin HS. 2009. The fundamental principles of financial regulation: 11th Geneva report on the world economy. Rep., Cent. Econ. Policy Res. London:
  15. Brunnermeier MK, Pedersen LH. 2009. Market liquidity and funding liquidity. Rev. Financ. Stud. 22:62201–38
     [Google Scholar]
  16. Campbell SD. 2006. A review of backtesting and backtesting procedures. J. Risk 9:1–17
     [Google Scholar]
  17. Chambers CP. 2009. An axiomatization of quantiles on the domain of distribution functions. Math. Finance 19:2335–42
     [Google Scholar]
  18. Cheridito P, Li T. 2009. Risk measures on Orlicz hearts. Math. Finance 19:2189–214
     [Google Scholar]
  19. Cherny A, Madan D. 2009. New measures for performance evaluation. Rev. Financ. Stud. 22:72571–606
     [Google Scholar]
  20. Christoffersen P. 1998. Evaluating interval forecasts. Int. Econ. Rev. 39:841–62
     [Google Scholar]
  21. Christoffersen P. 2010. Backtesting. Encyclopedia of Quantitative Finance R Cont New York: Wiley https://doi.org/10.1002/9780470061602.eqf15018
    [Crossref] [Google Scholar]
  22. Christoffersen PF, Pelletier D. 2004. Backtesting value-at-risk: a duration-based approach. J. Financ. Econom. 2:84–108
     [Google Scholar]
  23. Cont R, Deguest R, He XD. 2013. Loss-based risk measures. Stat. Risk Model. 30:2133–67
     [Google Scholar]
  24. Cont R, Deguest R, Scandolo G. 2010. Robustness and sensitivity analysis of risk measurement procedures. Quant. Finance 10:6593–606
     [Google Scholar]
  25. Costanzino N, Curran M. 2015. Backtesting general spectral risk measures with application to expected shortfall Work. Pap., Univ. Toronto and Bank of Montreal
  26. Daníelsson J, Jorgensen BN, Samorodnitsky G, Sarma M, de Vries CG. 2013. Fat tails, VaR and subadditivity. J. Econom. 172:2283–91
     [Google Scholar]
  27. Delbaen F 2002. Coherent risk measures on general probability spaces. Advances in Finance and Stochastics: Essays in Honour of Dieter Sondermann K Sandmann, PJ Schönbucher 1–37 New York: Springer
     [Google Scholar]
  28. Delbaen F, Bellini F, Bignozzi V, Ziegel JF. 2016. Risk measures with the CxLS property. Finance Stochast. 20:2433–53
     [Google Scholar]
  29. Denneberg D. 1994. Non-Additive Measure and Integral Boston: Kluwer Acad.
  30. Dhaene J, Goovaerts MJ, Kaas R. 2003. Economic capital allocation derived from risk measures. N. Am. Actuar. J. 7:44–59
     [Google Scholar]
  31. Duffie D, Pan J 1997. An overview of value at risk. J. Derivat. 4:37–49
     [Google Scholar]
  32. Duffie D, Pan J 2001. Analytical value-at-risk with jumps and credit risk. Finance Stochast. 5:2115–80
     [Google Scholar]
  33. Eeckhoudt L, Schlesinger H. 2006. Putting risk in its proper place. Am. Econ. Rev. 96:280–89
     [Google Scholar]
  34. Embrechts P, Wang B, Wang R 2015. Aggregation-robustness and model uncertainty of regulatory risk measures. Finance Stochast. 19:763–90
     [Google Scholar]
  35. Engelberg J, Manski CF, Williams J. 2009. Comparing the point predictions and subjective probability distributions of professional forecasters. J. Bus. Econ. Stat. 27:130–41
     [Google Scholar]
  36. Engle RF, Manganelli S. 2004. CAViaR: conditional autoregressive value-at-risk by regression quantiles. J. Bus. Econ. Stat. 22:367–81
     [Google Scholar]
  37. Fama EF, Miller MH. 1972. The Theory of Finance New York: Dryden
  38. Fissler T, Ziegel JF. 2016. Higher order elicitability and Osband's principle. Ann. Stat. 44:41680–707
     [Google Scholar]
  39. Fissler T, Ziegel JF, Gneiting T. 2016. Expected shortfall is jointly elicitable with value at risk—implications for backtesting. Risk Mag. 29:58–61
     [Google Scholar]
  40. Föllmer H, Schied A. 2002. Convex measures of risk and trading constraints. Finance Stochast. 6:4429–47
     [Google Scholar]
  41. Frittelli M, Gianin ER. 2002. Putting order in risk measures. J. Bank. Finance 26:71473–86
     [Google Scholar]
  42. Gaglianone WP, Lima LR, Linton O, Smith DR. 2011. Evaluating value-at-risk models via quantile regression. J. Bus. Econ. Stat. 29:1150–60
     [Google Scholar]
  43. Garcia R, Renault É, Tsafack G 2007. Proper conditioning for coherent VaR in portfolio management. Manag. Sci. 53:3483–94
     [Google Scholar]
  44. Gilboa I, Schmeidler D. 1989. Maxmin expected utility with non-unique prior. J. Math. Econ. 18:2141–53
     [Google Scholar]
  45. Gneiting T. 2011. Making and evaluating point forecasts. J. Am. Stat. Assoc. 106:494746–62
     [Google Scholar]
  46. Gordy MB. 2003. A risk-factor model foundation for ratings-based bank capital rules. J. Financ. Int. 12:3199–232
     [Google Scholar]
  47. Gordy MB, Howells B. 2006. Procyclicality in Basel II: Can we treat the disease without killing the patient?. J. Financ. Int. 15:3395–417
     [Google Scholar]
  48. Haas M. 2005. Improved duration-based backtesting of value-at-risk. J. Risk 8:17–38
     [Google Scholar]
  49. Hampel FR. 1971. A general qualitative definition of robustness. Ann. Math. Stat. 42:61887–96
     [Google Scholar]
  50. Hansen LP 2013. Challenges in identifying and measuring systemic risk. Risk Topography: Systemic Risk and Macro Modeling M Brunnermeier, A Krishnamurthy 15–30 Chicago: Univ. Chicago Press
     [Google Scholar]
  51. Hansen LP, Sargent TJ. 2007. Robustness Princeton, NJ: Princeton Univ. Press
  52. Hart HLA. 1994. The Concept of Law Oxford, UK: Clarendon. , 2nd ed..
  53. He XD, Jin H, Zhou XY 2015. Dynamic portfolio choice when risk is measured by weighted VaR. Math. Oper. Res. 40:3773–96
     [Google Scholar]
  54. He XD, Peng X. 2018. Surplus-invariant, law-invariant, and positively homogeneous acceptance sets must be induced by value-at-risk. Oper. Res. 66:51268–75
     [Google Scholar]
  55. Heyde CC, Kou S. 2004. On the controversy over tailweight of distributions. Oper. Res. Lett. 32:399–408
     [Google Scholar]
  56. Heyde CC, Kou SG, Peng XH 2006. What is a good risk measure: bridging the gaps between data, coherent risk measures, and insurance risk measures. Work. Pap., Columbia Univ. New York:
     [Google Scholar]
  57. Holzmann H, Eulert M. 2014. The role of the information set for forecasting—with applications to risk management. Ann. Appl. Stat. 8:1595–621
     [Google Scholar]
  58. Hong CS, Herk LF. 1996. Increasing risk aversion and diversification preference. J. Econ. Theory 70:180–200
     [Google Scholar]
  59. Huber PJ. 1981. Robust Statistics New York: Wiley
  60. Huber PJ, Ronchetti EM. 2009. Robust Statistics New York: Wiley. , 2nd ed..
  61. Hull J. 2009. Risk Management and Financial Institutions New York: Prentice Hall. , 2nd ed..
  62. Ibragimov R. 2004. On the robustness of economic models to heavy-tailedness assumptions Tech. Rep., Yale Univ. New Haven, CT:
  63. Ibragimov R. 2009. Portfolio diversification and value at risk under thick-tailedness. Quant. Finance 9:5565–80
     [Google Scholar]
  64. Ibragimov R, Walden J. 2007. The limits of diversification when losses may be large. J. Bank. Finance 31:82551–69
     [Google Scholar]
  65. Jorion P. 2007. Value at Risk: The New Benchmark for Managing Financial Risk Boston: McGraw-Hill. , 3rd ed..
  66. Jouini E, Meddeb M, Touzi N. 2004. Vector-valued coherent risk measures. Finance Stochast. 8:4531–52
     [Google Scholar]
  67. Kahneman D, Tversky A. 1979. Prospect theory: an analysis of decision under risk. Econometrica 47:263–91
     [Google Scholar]
  68. Kerkhof J, Melenberg B. 2004. Backtesting for risk-based regulatory capital. J. Bank. Finance 28:1845–65
     [Google Scholar]
  69. Kijima M. 1997. The generalized harmonic mean and a portfolio problem with dependent assets. Theory Decis. 43:71–87
     [Google Scholar]
  70. Koch-Medina P, Moreno-Bromberg S, Munari C. 2015. Capital adequacy tests and limited liability of financial institutions. J. Bank. Finance 51:93–102
     [Google Scholar]
  71. Koch-Medina P, Munari C. 2016. Unexpected shortfalls of expected shortfall: extreme default profiles and regulatory arbitrage. J. Bank. Finance 62:141–51
     [Google Scholar]
  72. Koch-Medina P, Munari C, Šikić M. 2017. Diversification, protection of liability holders and regulatory arbitrage. Math. Financ. Econ. 11:63–83
     [Google Scholar]
  73. Kou SG, Peng X. 2016. On the measurement of economic tail risk. Oper. Res. 64:51056–72
     [Google Scholar]
  74. Kou SG, Peng X, Heyde CC. 2013. External risk measures and Basel Accords. Math. Oper. Res. 38:3393–417
     [Google Scholar]
  75. Krätschmer V, Schied A, Zähle H. 2014. Comparative and qualitative robustness for law-invariant risk measures. Finance Stochast. 18:2271–95
     [Google Scholar]
  76. Kupiec P. 1995. Techniques for verifying the accuracy of risk management models. J. Derivat. 3:73–84
     [Google Scholar]
  77. Kusuoka S. 2001. On law invariant coherent risk measures. Adv. Math. Econ. 3:83–95
     [Google Scholar]
  78. Lai TL, Shen D, Gross S. 2011. Evaluating probability forecasts. Ann. Stat. 39:52356–82
     [Google Scholar]
  79. Lambert NS, Pennock DM, Shoham Y. 2008. Eliciting properties of probability distributions. Proceedings of the 9th ACM Conference on Electronic Commerce, EC '08129–38 New York: ACM
     [Google Scholar]
  80. Liu F, Wang R. 2021. A theory for measures of tail risk. Math. Oper. Res. 46:31109–28
     [Google Scholar]
  81. Lopez JA. 1999a. Methods for evaluating value-at-risk estimates. Fed. Res. Bank San Francisco Econ. Rev. 2:3–17
     [Google Scholar]
  82. Lopez JA. 1999b. Regulatory evaluation of value-at-risk models. J. Risk 1:37–64
     [Google Scholar]
  83. Maccheroni F, Marinacci M, Rustichini A. 2006. Ambiguity aversion, robustness, and the variational representation of preferences. Econometrica 74:61447–98
     [Google Scholar]
  84. McMinn RD. 1984. A general diversification theorem: a note. J. Finance 39:541–50
     [Google Scholar]
  85. McNeil AJ, Frey R. 2000. Estimation of tail-related risk measures for heteroskedastic financial time series: an extreme value approach. J. Empir. Finance 7:271–300
     [Google Scholar]
  86. McNeil AJ, Frey R, Embrechts P. 2005. Quantitative Risk Management: Concepts, Techniques, Tools Princeton, NJ: Princeton Univ. Press
  87. Moscadelli M. 2004. The modelling of operational risk: experience with the analysis of the data collected by the Basel Committee Work. Pap., Banca d'Italia Rome:
  88. Osband KH. 1985. Providing incentives for better cost forecasting PhD Thesis, Univ. Calif. Berkeley:
  89. Rockafellar RT, Royset J, Miranda S 2014. Superquantile regression with applications to buffered reliability, uncertainty quantification, and conditional value-at-risk. Eur. J. Oper. Res. 234:1140–54
     [Google Scholar]
  90. Rockafellar RT, Uryasev S. 2002. Conditional value-at-risk for general loss distributions. J. Bank. Finance 26:71443–71
     [Google Scholar]
  91. Ruszczyński A, Shapiro A. 2006. Optimization of convex risk functions. Math. Oper. Res. 31:3433–52
     [Google Scholar]
  92. Samuelson PA. 1967. General proof that diversification pays. J. Financ. Quant. Anal. 2:1–13
     [Google Scholar]
  93. Savage LJ. 1971. Elicitation of personal probabilities and expectations. J. Am. Stat. Assoc. 66:336783–810
     [Google Scholar]
  94. Schmeidler D. 1986. Integral representation without additivity. Proc. Am. Math. Soc. 97:2255–61
     [Google Scholar]
  95. Schmeidler D. 1989. Subjective probability and expected utility without additivity. Econometrica 57:3571–87
     [Google Scholar]
  96. Shi Z, Werker BJM. 2012. Short-horizon regulation for long-term investors. J. Bank. Finance 36:123227–38
     [Google Scholar]
  97. So MKP, Wong CM. 2012. Estimation of multiple period expected shortfall and median shortfall for risk management. Quant. Finance 12:5739–54
     [Google Scholar]
  98. Song Y, Yan JA 2006. The representations of two types of functionals on and . Sci. China Ser. A 49:1376–82
     [Google Scholar]
  99. Song Y, Yan JA. 2009. Risk measures with comonotonic subadditivity or convexity and respecting stochastic orders. Insurance Math. Econ. 45:3459–65
     [Google Scholar]
  100. Staum J. 2013. Excess invariance and shortfall risk measures. Oper. Res. Lett. 41:147–53
     [Google Scholar]
  101. Tasche D. 2002. Expected shortfall and beyond. J. Bank. Finance 26:71519–33
     [Google Scholar]
  102. Thomson W. 1979. Eliciting production possibilities from a well-informed manager. J. Econ. Theory 20:3360–80
     [Google Scholar]
  103. Tversky A, Kahneman D. 1992. Advances in prospect theory: cumulative representation of uncertainty. J. Risk Uncertainty 5:297–323
     [Google Scholar]
  104. Von Hippel PT. 2005. Mean, median, and skew: correcting a textbook rule. J. Stat. Educ. 13:21–13
     [Google Scholar]
  105. Wang R, Zitikis R 2021. An axiomatic foundation for the expected shortfall. Manag. Sci. 67:31413–29
     [Google Scholar]
  106. Wang SS, Young VR, Panjer HH 1997. Axiomatic characterization of insurance prices. Insurance Math. Econ. 21:2173–83
     [Google Scholar]
  107. Weber S. 2006. Distribution-invariant risk measures, information, and dynamic consistency. Math. Finance 16:2419–42
     [Google Scholar]
  108. Weber S, Anderson W, Hamm AM, Knispel T, Liese M, Salfeld T. 2013. Liquidity-adjusted risk measures. Math. Financ. Econ. 7:169–91
     [Google Scholar]
  109. Wen Z, Peng X, Liu X, Bai X, Sun X. 2013. Asset allocation under the Basel Accord risk measures Work. Pap., Peking Univ. Beijing, China:
  110. Xi J, Coleman TF, Li Y, Tayal A 2014. A gradual non-convexification method for minimizing VaR. J. Risk 16:23–47
     [Google Scholar]
  111. Xia J. 2013. Comonotonic convex preferences. Work. Pap., Acad. Math. Systems Sci., Chinese Acad. Sci. Beijing:
  112. Yaari ME. 1987. The dual theory of choice under risk. Econometrica 55:95–115
     [Google Scholar]
/content/journals/10.1146/annurev-statistics-030718-105122
Loading
  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error