judges are tougher before the lunch break than in the morning or after lunch judges are tougher on monday after football team loses in the weekend more leniency on defendant's birthday in France fewer asylims are given on a hot day inter-judge differences: female judges, judges appointed by democrats are more lenient insurance, doctors, fingerprint matching, xray examination singular decision evaluative vs. predictive judgements intra-personal vs. interpersonal judgements unverifiable judgements judgement process and outcome bias and noise, mean squares method (penalizes greater deviation) bias contribures as much as noise, contributions are independent level noise: difference in judges pattern noise: difference in judging cases occasion noise: variability due to transient causes (weather, mood) - judge disagrees with himself asking himself a second time may be almost as good as getting a second opinion decision/judging is subconscious good mood people are vulnerable to first impressions and stereotypes, cooperative, easy to negotiate with gamblers fallacy: previous case acts as frame of reference, judges try to balance decisions against streaks (sequence effect) stock market performs better on a sunny day doctors are more likely to prescribe opioids for pain at the end of the shift stress and fatigue influence judgement system noise is usually a lot greater than occasion noise brain variability to large degree is affected by brain function itself - cant easily be controlled Social groups amplify noise. Social influence introduces noise. Early opinion makes dispriportionate impact. Crowds are "wiser" when their views are independent. Informational cascade. Social pressure cascade. group polarization, similar to cascade effect percent concordance (PC) correlation coefficient multiple regression vs. clinical judgement (humans). weighted average is better than humans: there is so much noise in jusging that noise-free model of a judge outperforms the judge itself you dont need models more precise than data tetlock: the average expert is just as accurate as a dart-throwing chimpansee objective ignorance - unpredictable behavior affecting outcome. human explanations of events are through backward search for causes, vs. statistical thinking magican number 7 - # of categories that humans can classify on intensity scale before comparing head-to haed without making errors (system 1 thinking) pattern noise is the larger component of the system noise respect-experts GMA debiasing ex post/ex ante decision hygene delfi-method bryer score: rewards high calibration (good average) high resolution (greater differentiation between forecasts) applies to probabilistic forecasts 2% superforecasters. Nit necessarily intelligence community psychological bias vs. statistical bias. Psychological may not show as statistical but rather as nois since it tugs in multiple directions. pneumonia diagnosis by radiologists: variation in skill explains 44% of variation diagnostic decisions. angiogram 70% blocked - need stent, endometrionsis - laproscopy camera, dramatic disagreement between gyn surgeons on whether endo 3 camera views TB diagnosis variability, between radiologists of different countries melanoma diagnosis by pathologists, 64% agreement, 38% false negatives radiologists breast cancer false negatives vary diff. radioologistfrom 0 to 50%, false pos. <1% to 62% occasion noise: docs are running behind later in the day and before lunch: more noise 2 psychiatrists highly trained specialist agree whether the patient has a major depressive disorder between 4% -15% of the time performance evaluations 75% is system noise anchors, forced rankings, occasionally counterproductive: why rank? strategic ranking hiring: unstructured interviews are nearly useless wrt future job performance noise as a vehicle for change: more adaptible, algorithms may be gamed rules vs standards/guidelines in case of noise - movement towards rules. Rigid rules are not followed bureaucratic justice cost of decisions for computation-judgement-taste unverifiable judgements