The Perils of Simplifying Risk To a Single Number 286
A few weeks back we discussed the perspective that the economic meltdown could be viewed as a global computer crash. In the NYTimes magazine, Joe Nocera writes in much more depth about one aspect of the over-reliance on computer models in the ongoing unpleasantness: the use of a single number to assess risk. Reader theodp writes: "Relying on Value at Risk (VaR) and other mathematical models to manage risk was a no-brainer for the Wall Street crowd, at least until it became obvious that the risks taken by the largest banks and investment firms were so excessive and foolhardy that they threatened to bring down the financial system itself. Nocera explores the age-old debate between those who assert that the best decisions are based on quantification and numbers, and those who base their decisions on more subjective degrees of belief about the uncertain future. Reliance on models created a 'false sense of security among senior managers and watchdogs,' argues Nassim Nicholas Taleb, who likens VaR to 'an air bag that works all the time, except when you have a car accident.'"
Gladwell's "Blowing Up" (Score:5, Interesting)
For an EXCELLENT article about this, read Malcolm Gladwell's "Blowing up", which you can find online for free:
http://www.gladwell.com/2002/2002_04_29_a_blowingup.htm [gladwell.com]
Welcome to the Age of Bayes (Score:4, Interesting)
Nocera explores the age-old debate between those who assert that the best decisions are based on quantification and numbers, and those who base their decisions on more subjective degrees of belief about the uncertain future.
Why is anyone still making this distinction, as we now know that the only self-consistent numerical representation of risk follows directly from our subjective degrees of belief about the uncertain future? Furthermore, we have known this for over a generation... isn't it about time that the knowledge start filtering into the popular discourse?
While Bayesian methods are not always all that useful for practical problems (I use them on occasion in my work) the conceptual foundations and deeper understanding of the nature of plausible reasoning and its relation to probability theory needs to be more widely understood.
One of the big take-home messages from the Bayesian revolution is that probability theory is nothing but quantification of what we do subjectively, insofar as our subjective impressions are self-consistent, so the only people who are still debating quantitative vs subjective approaches as such are people who do not understand the question.
Re:Taleb goes much farther than that (Score:2, Interesting)
So Mr. Mathematically-savvy Man, why don't you go ahead and transform economics for the better? I'm sure there are many more "obvious" things out there to come up with.
VaR is a pretty decent risk measure on a micro scale. The real problem with it is that VaR constraints tend to make banks less diversified, introducing systemic risk. When things go sour, banks are forced to sell off similar assets, and because all of the banks tend to hold assets with similar risk, markets fluctuate all the more.
It is telling that a broad index of hedge funds is better resistant against risk than an index of banks.
It's simpler than that (Score:5, Interesting)
Risk models are largely irrelevant because the only risk anyone in the financial sector is really interested in minimizing is the risk that they will get fired. The way to do that is to do almost exactly the same thing as everybody else, no matter how mind blowing stupid it is. Plenty of people realized that banks etc were not nearly as sound as commonly believed years ago. Those that tried to act on this were fired long ago since they weren't making as high a ROI as those willing to invest in dodgy hedgefunds etc. Rational market my ass.
Open Source Risk Modeling (Score:5, Interesting)
Re:Math? (Score:5, Interesting)
After seeing the rampant fraud committed by the global financial elite, I'm very inclined to agree with you. What we need isn't just a number that quantifies risk, but also a number that quantifies trust.
I would pay for a service that tracks every person involved in business that was ever convicted, under indictment, or subject of a complaint. It should also track which firms employed them and where they are working now. It should also cover which "civil servants" were "on watch" at the time.
Taleb doesn't know everything (Score:5, Interesting)
As an example, suppose that the distribution suggests the chance of losing 50 million dollars is +3 sigma for some measure. The problem is that there is a subtle effect - say panic, herd effect or some interaction of derivative models - which only becomes significant around the 3 sigma mark. The result could be that the exposure at a 4 sigma event is billions of dollars. A proper risk model would need to take this into account
My conclusion based on what I have read so far is that the physicists (in particular) involved in developing quantitative models would have benefited from a lot more exposure to real world experiment. They would then have had more of a clue about the unreliability of data away from the mean, scatter, and the importance of the fact that in physics subtle errors turn out to be signs that the model is wrong - e.g. relativistic effects only become important at a significant fraction of c.
Model accuracy wasn't the only problem (Score:5, Interesting)
The most accurate model in the world won't help if you don't pay atention to the results it produces.
There's also apparently an issue with the classical VaR models depending on transparent pricing, which these real estate instruments lack. So some of the most troublesome assets apparently weren't in the model.
Re: I don't think (Score:5, Interesting)
You might think that if you have three investments with a 10% risk of losing £1,000,000 the chances of all three of them losing £1,000,000 is 0.1*0.1*0.1 = 0.001 or 0.1%.
No, no-one who actually calculates and uses VaR thinks that. Anyone who has done any statistics, like all finance quants, will correctly take into account covariances. The actual problem is the interpretation of the "correct" VaR, and relying on it too heavily.
I'll give you the actual definition of VaR. If you calculate the VaR(10 day, 5%) to be $100,000, this means that there is a 5% chance that the loss on your portfolio over a 10 day period will be larger than $100,000, or that your profit will be larger than $100,000 assuming a symmetric distribution. It's when people think "Oh that's great, we can ONLY lose $100,000" when you have a problem. The actual loss could be ANY value larger than $100,000.
It's hardly a perfect statistic, since there are still many assumptions involved. However, it's still a decent estimator and it's better than making a wild guess based on gut feelings. Despite what most people currently believe, a lot of brainpower has gone into developing financial theories and some stuff is pretty damn good. The financial industry deserves some bashing, but it frustrates me when people spread incorrect information; at least complain about the right things.
Re:Math? (Score:3, Interesting)
Ah, see, that doesn't work either. When the market moves against the wrong players they'll use their political influence and get the rules changed. Many hedge funds and others who were 'correct' eventually lost out anyways, as the Fed simply prints money to fill the holes for the right people.
Fundamentally large parts of the market should be liquidated and shut down; overcapacity is rampant and consumers do not want the products in question at the prices they can be produced, the demand that seemed to be there was just a reflection of demand when money was free.
Do you want to bet that will be allowed to happen? Or do you want to be that the Fed and government will simply confiscate your resources through printing and borrowing and keep their friends in the green at your expense, without you having any say?
You can be the most rational market player in the world, yet you can't win when they'll change the rules at whim and confiscate any profit you make either way. The only way to even avoid losing money is to move them elsewhere and not participate in the rigged markets. Not that it's easy to find free and unmanipulated markets; guess why 'coordinated action' has come into such a vogue; debase all currencies and savers cant protect their money.
Models (Score:3, Interesting)
Reality's an untamed beast
That's difficult to master,
But models are quite docile
And give you answer faster.
From a pome I saw in a computer book from the 70's, can be found online here http://www.langston.com/Fun_People/1993/1993AFE.html [langston.com]
Re:Minmaxing ftw! (Score:2, Interesting)
The problem is that your MinMaxers typically look for odd corner cases where multiple rules add up to more than average results. They also exploit Grey Areas where rules are under defined. This gives a decided edge to the MinMaxer over the other players, now while you're all working together a Win for one person should be a Win for the Team, it is not in fact a Win.
The MinMaxer succeeds more often, does more, enjoys more (maybe) at the expense of other players fun.
To take D&D for an example, I have a character that for a while exploited the Spiked Chain a weapon with a Long Reach that can Trip targets and get a free attack the target tries to stand up again. I got this trick from a GM complaining in a Forum about 2 Chain users in his group meat grinding everything he threw at them. Because of the reach of the chains, the opponents never got into melee range to do any damage. So at this point it is an impervious and one sided situation for the MinMaxers.
They may or may not have had Combat Reflexes to allow them to take multiple free attacks on Targets coming into their range, or getting up from trips.
So I combine this Chain + Combat Reflexes + Protection from Arrows (no stand off engagements forced enemies into my well defined corner case) I combined this with a High Dexterity (more free attacks) and Three Attack Dogs trained to Attack and Trip opponents on command. This gave me more opportunities to use the Free attacks on Tripped Opponents that stood up, in addition to any incidental damage the Dogs actually did with their attacks.
Now take ANY other character in the party that is designed for melee combat, and ask him why he bothered to show up. At first level he'd get 1 attack, I could have up to 8 (1 Normal, 4 AoOs, 3 Dogs)
The Dogs exploited a Grey area in the rules, nowhere does it say how many Dogs you can say Attack to in one round, its left to GM discretion. The GM for the most part will try his best not to screw his players out of legitimate uses of their abilities while not letting his game run away from him.
I do crowd control nowadays while the real melee characters kill bosses. Everybody in their niche, its more fun for everybody, and shows that we work as a team, not just one guy trying to be a superstar.
Robin Laws has a good section in his book on Game mastering on figuring out what your players want and giving it to them. This was also figured out long ago for MUDs how some players like to be PvP or PvE long before MMOs came on the scene.
Needless to say no player 'wants' to sit around and watch other people having fun, while feeling impotent and ineffective.
The other problem with MinMaxers is that in their search for corner cases, they will search every book for crunchy bits to exploit. This tilts the game in favor of people with no life and or lots of money to buy source books over casual or thrifty gamers.
This goes against a principle of fair play in RPGs that says anyone CAN has the same choices and same chances to succeed or fail. Everything else should be the luck of the dice.
Creative players should be rewarded, other players will talk forever about some cool idea one of their buddies had, not what esoteric combo of rules he used to unhinge the game.
I think that's all I can say on this.
Re:Not an Either/Or Situation (Score:3, Interesting)
You'd have to provide some evidence that most foreclosures are investment properties. More likely everyone believed that they'd be able to refinance out of their ARM on their primary (read:only) house, because "home values all ways go up." When that wasn't the case you get what we see now.
There's plenty of blame to be spread around, from the builders who overbuilt saturating the market to the bankers who financed every subdivision to come along, to the home buyers who thought they wouldn't really have to pay the higher rate in x years, to the mortgage brokers who sold loans to people who couldn't afford them on commission, to the banks who bought, them bundled, them broke them apart, and took out CDSs on them, to the hedge funds, retirement pensions, and private investors and anyone else who didn't bother to divide median home price by median income to see if their investments were reasonable.
Re:Math? (Score:3, Interesting)
Sorry, you aren't putting your money into companies by gambling on the stock market. If you want to put money into a company you have to buy stock directly from them. Yes, there are occasions where companies do release new stock which is when money gets invested into actual production, but most of the stock market and other financial markets are zero sum speculations that serve little purpose other than to enrich those with the best insider information and skill.
The whole idea that investing on the stock market is always good is bogus. There are very rarely any cheap/undervalued stocks nowadays. Sure, stocks go up in price which make them look like they were cheap before, but that is usually an illusion because one idiot is hoping that another idiot will buy the stock at an even higher price. Speculation value is only artifical value that is zero sum in the long run. Dividends and company assets is what really matters when evaluating the true value of stocks.
It gets worse. With the lack of information about companies due to lax book keeping regulations, people looking to actually buy stock based on real value can't, because it is near impossible to evaluate companies as there is a lot of debt and bad asset hiding going on. Finally, with all the insider bailouts going on, even if you had an idea of what the company had on the books you still couldn't evaluate the company, because the goverment might just decided to prop up that specific company.
To be fair, not all companies are equally bad. But I am talking about the attitude of the financial markets in general that has turned into little more than self serving beasts.
Re:Welcome to the Age of Bayes (Score:3, Interesting)
How the hell can you apply any kind of probability measure to a self-aware environment like a marketplace?
Bayesian methods or any other are not going to get around the way the very measure of the risk is going to alter the market itself. You can't use physics and math to predict biologic and cultural processes, not when the processes have the same order of complexity as entire ecosystems and a capacity to learn and change that we haven't yet even begun to understand.
Introduce a risk management tool into a real estate market so you can enlarge that market by identifying marginal mortgage situations that are actually safe enough to go with, and you create whole new market segments building new housing to meet this brand new demand. And speculating in old housing; taking out high risk mortgages to flip a house and sell it at a hefty profit in 12 months, rinse and repeat. Watch those new market segments grow, and distort and inflate the entire housing industry.
We been there. We done that. Let's not do it again.
We need to recognise that any predictive market tools like those of risk management can have a profound, immediate, and unpredictable effect on the underlying market the purport to represent.
Re:Math? (Score:4, Interesting)
You're assuming that the bet you're making when entering the stock market is "The price per share of the stock in [insert company here] will go up before I have reason to sell my shares." If that's the way you want to bet, fine - but you'd be an idiot to bet that way. You should be entering the stock market with this bet: "The combined value of change in price of the stock plus the dividends paid will be more than the value of what I paid for the stock." Note that I mentioned only value, not price. Although money has been described as "the universal symbol for value received", most currencies in use at this point are fiat currencies that have no fixed value, either in non-fiat currencies or in commodities. Therefore, what costs $1 today might cost $10 a week later. (In fact, Zimbabwe's economy has been doing this kind of thing recently.)
So, depending on how the rest of the economy changes - buying a stock at $100/share and selling it a year later at $10/share might actually be a good idea - if the stock paid out $95/share in dividends and the economy is otherwise unchanged, or if that $10 will buy more than $100 would have a year earlier.
Re: I don't think (Score:3, Interesting)
Disclaimer: IAAMFPHDS (I am a mathematical finance PhD student).
While quants could accurately gauge the historical covariance of different assets in a portfolio, what they failed to take into account is that there is correlation in the tails of the distribution.
An example of this is that, back in the good old days, there was a degree of correlation between the Dow and the FTSE 100. If the FTSE 100 went up, it was a decent indicator that the Dow would also be up, but by no means a sure thing. However, during the crisis, the two indices practically moved in lockstep.
The moral of the story is that in the rare event that things get bad, correlation tends to spike. The models failed to take account of this, which is part of the reason we're in this mess.
That's not the point either. (Score:4, Interesting)
If Mr A gave Mr B billions of dollars of The Public's Money to play at a casino and both Mr A and Mr B got filthy rich when times were good, and when it blows up all that happens is Mr B loses his job and Mr A keeps his job by blaming Mr B or saying BS like "perfect storm/everyone was doing it".
Why then should Mr A and Mr B be doing things differently?
After all, in the following year, Mr A passes billions to Mr C who does pretty much the same thing as Mr B. And Mr B? He's hired by Mr D who wants Mr B to make him richer (just like he did for Mr A).
AFAIK, not long after LTCM blew up, its founder John Meriwether still managed to get hundreds of millions of dollars to start a hedge fund.
What I see are individuals making pretty rational decisions, those decisions sometimes just happen to be bad for a lot of other people. But why should those individuals care?
Their conscience should bother them? The last I checked the Economists leave the conscience stuff to "The Invisible Hand". People laugh at the religious, guess who really has even less of a clue on how things work? At least the religious have some idea about the "Invisible Hand" sort of stuff.
It's hilarious that you have all those people saying/writing stuff like "When Genius Failed".
That's like the sheep saying the wolves have failed just because the wolves dropped 95% of a billion sheep over a cliff, whilst "only" managing to stuff themselves to the brim with 1% of the billion sheep. I'm sure the wolves were a bit upset about the whole thing, but hey there are billions more sheep...
Yeah I see failure alright. Go figure where.
You want to reduce the risk of stuff blowing up, and how big they blow up? It has nothing to do with creating better financial models or better economic theories.
It has to do with making and enforcing rules like: if too many sheep die, we shoot and skin the wolves responsible. Simple as that.
All that transparency and regulation is worthless if at the end of the day the wolves get away.
Systems are always gamed (Score:3, Interesting)
> After seeing the rampant fraud committed by the global financial elite, I'm very inclined to agree with you. What we need isn't just a number that quantifies risk, but also a number that quantifies trust.
Won't they just game that number, too? Once you fix the rules for any system, people will start to attack them. And, based on what I've seen in online games, they'll find a way to break them. Especially when there's real money at stake, not just virtual gold and items (though even those can be converted to cash these days...).
This is related to the paradox of high standards: once you set your standards too high, the only people who "meet" them are the worst cheaters.
Re:That's not the point either. (Score:3, Interesting)
Except the sheep and wolf analogy has an implied zero-sum notion to it. From our fairy tales and such we think of the wolves as nasty evil creatures, but seriously look at nature: they fulfill a role in the ecosystem and are simply a part of it.
They way I would answer your question is to say we are all wolves and we are all sheep, and the analogy is simply way off mark. Do you seriously think it wise to micro-manage every aspect of the entire economy according to some predefined notion of fairness? Because that sounds like what you are implying. Its a well worn argument but I'll say it anyway: It didn't work for the Soviets, and it didn't work for the Chinese.
The real problem as I see it is already perfectly summed up in Feynmans famous appendix to the Challenger report [fotuva.org]. The real problem is we have developed highly complex financial systems that more or less require a postgraduate degree in mathematics to understand, I'm willing to bet the vast majority of people making the decisions (not the models) that led us into the current crisis had little to no real understanding of what they were dealing with, and as such they exaggerated the (lack of) risk to suit their managerial requirements. Precisely as occurred in the Challenger disaster and precisely as Feynman went to great pains to describe.
There is no vast conspiracy of criminal selfishness, no wolves and sheep, just a bunch of people working with complex mathematical constructs and absolutely no idea of how they work. The sort of PHB fuckup that most slashdotters should readily understand.