---start--- epidemiology 1/19/98 Gary Smith there was a power outage at NBC today so they couldn't get the handout copied for us, so we do not have a handout for today's lecture. Oh joy, oh happiness, that is one less thing to schlep around in my backpack. [allegedly amusing anecdote snipped again] ok. he says that math has logic to it like biology does. (a+b)(a+b) = a^2 + 2ab + b^2 how do you know that's true? teacher says so. but really, how do you know? well, the Greeks proved it geometrically. imagine a line of length a |-------| add a line of lenghth b |-------|----| then you can square that by making that one line into the four sides of a square. then if you draw lines through the square, you see that you have one square that is bxb, and one that is axa, and two rectangles that are axb, so that proves it. now, in general, almost no one fails epidemiology. so this morning will be a real lecture...then, this afternoon, we will be having a review session instead of a lecture. today's topic is sensitivity and specificity: problems with cohort and case-control tsudies: 1. identifying cases 2. bias and validity 3. confounding remember, we've been going over the things that can ruin your study. bias, a systematic error; and validity, the extent to which a study measures what it is supposed to - a valid study is one without bias. we talked about confounding for a while. there is also something else that can introduce bias - that's a systematic problem with identifying cases. all epidemiological activity involves seeing and counting diseased animals. but you know, it is not easy to know for sure if an animal has the particular disease. misdiagnosis occurs. the problem of misdiagnosis: it's easy to imagine a clinician looking at a healthy dog and saying "it has diabetes." This is a false positive diagnosis. also, a clinician may look at an animal that actually has diabetes, and say "it is in fact healthy" and that would be a false negative diagnosis. we can represent this in a table form: test results disease status + (sick) - (healthy) positive A B negative C D total A+C B+D sensitivity = A/(A+C) specificity = D/(B+D) now, think about our bad clinician and imagine a gold standard which allows us to divide the patients into truly sick and truly well - those are the columns. so, of all the truly ill patients, the clinician diagnosed A of them, and of those that were actually healthy, the clinician diagnosed D of them. these are the correct diagnoses. now, the "diseased" column, eg the + column, is used for sensitivity calculations. A animals test positive out of the total of A+C animals which actually have the disease. sensitivity is the proportion of true positives detected. conversely, specificity deals with what happens in the healthy, - column. it's D, the number of true negative animals identified, out of B+D, the total number of actual healthy, negative animals. right now we're applying sensitivity and specificity to someone's skill. it also is applied to specific tests. a valid test must be sensitive and specific. note: epidemiological sensitivity isn't the same as immunlogical sensitivity. immunological sensitivity: the ability to detect small amounts of substance epidemiological sensitivity: the proportion of diseased animals that test positive. you have to learn this. there is no way around it. oy, again with the learning today. sigh. so, sensitivity is important - if an animal is misclassified wrt if it is diseased or not, this alters RR or OR, and may obscure a true association, or enhance a spurious association. a good diagnostic test must be highly specific and highly sensitive. however, many tests do not have this. often, sensitivity and specificity vary inversely. this is true of many diagnostic tests that depend on measuring some constituent of the serum. slide - probability distributions of proportions of animals with a certain amount of substance x in their serum. we see that in diseased animals, the curve is skewed to the right. however, x is found in both healthy and diseased animals, and differs only in average concentration. so the cutoff point that determines if an animal is diseased or not, is pretty arbitrary. we see that there are a number of false positives and a number of false negatives. if we move the cutoff point to decrease the false positives, we then increase the false negatives. this shows the inverse relationship b/w specificity and sensitivity. in practice, your choice of test may depend on a number of things. what is the cost of a false positive? uncertainty, anxiety, perhaps an expensive or dangerous therapy. what about false negative? cost of additional workup, waste of time, waste of money, perhaps spread of disease. so what do you focus on? specificity or sensitivity? many tests, esp those involving serum, have this reciprocal relationship b/w sensitivity and specificity. so, you choose a test w/high sensitivity when the penalty of false positive result is low and you have to minimize your chance of missing the true diagnosis. you choose a test with high specificity when the penalty of false positives is high, a dangerous or expensive therapy. apparent and true prevalence: apparent prevalence, that you measure, is always different from true prevalence. test result diseased healthy diseased A B healthy C D if N = A+B+C+D apparent AP = (A+B) /N note that this apparent prevalence includes false positives and misses some true positives. true P = (A+C) /N AP and TP will only be the same if you have a perfect test, or if the number of FP and FN is identical (eg, B=C) by some accident. actually, B is usually much larger than C, so apparent prevalence is often greater than real prevalence. P = (AP + specificity - 1) / (sensitivity + specificity - 1) don't memorize formula. know that if you know the sensitivity and specificity of a test, you can correct the apparent prevalence, AP. Predictive value of a test: focusing on positive predictive value, PPV this is the proportion of diseased animals among those that test positive actual status test result + - diseased A B healthy C D PPV = A/(A+B) this is probably the most important clinical characteristic of the test. the PPV reflects the probability of a positive test really meaning that the patient has a disease. eg, PPV = 100% means that all + results mean the patient has the disease. the predictive value increases with prevalence. therefore, as an eradication program progresses, you will need a test with progressively higher and higher specificities and sensitivities. this is important in two contexts. as you become more successful in your eradication program, fewer and fewer of those positives that turn up in your screening procedure will actually be positive. now, if you're culling your positives, you are now killing more and more healthy animals, which is a bad thing from many points of view. also, the other thing is that because PPV decreases with prevalence, it has a considerable impact on how much attention you pay to test results when considering a single animal. consider a human being. often, a newspaper article will describe a disease, and then everyone runs to the doctor requesting the test for the disease. let's take prostate cancer. there was a test for a while with low sensitivity and low specificity. if you had a young man under 40 with no clinical signs who demanded the test, and then tested positive, what did you do? well, you'd think that the test was probably wrong. his demographic group has a lot of false positives, and likelihood of someone in his group having prostate cancer is very very low. PPV of the test in this group is very very low. the proportion of positive results that really do predict disease is low. now, if a 70 yr old man with a suspicious nodule asked for the test, and then tested positive, you would be more likely to think the positive result does actually reflect the disease state. b/c this demographic group is much more likey to have the disease. prevalence in this group is high. PPV is high. so if you get someone showing up at your office with a dog and they want a test done on it, and the dog is in a population very unlikely to have the disease, and then you get a positive result, what do you tell the owner? you have to explain that it is probably a false positive, b/c the prevalence in that group is very very low, and PPV increases or decreases as prevalence increases or decreases. so, at the end stages of an eradication program you have to use a test with a very very very high sensitivity and specificity - which is probably a more expensive test. -------- Review session, 1-2 pm format for 2x2 tables - you see a lot of these in epidemiology. you should use his format because it's harder to make mistakes. First heading (labels columns)----------> Disease + (sick) - (well) ci row labels: characteristic yes A B A/A+B an exposure, or something no C D C/C+D So your columns are going to be + (sick) or - (healthy) your rows are going to be yes or no (exposed, unexposed) ci (yes) = A/A+B ci (no) = C/C+D RR = A/(A+B) OR = AD/CB ------- **must know the RR formula!*** C/(C+D) RR is measured in cohort studies only. These are where you start with healthy animals and wait for them to get sick. you divide them into two groups, one which isn't exposed to your risk factor, and one which is, and then you track disease occurrence. 1. get a group 2. divide by risk factor 3. find cases with case control study, you start with your cases. then you select your controls (your non-cases), and use some arbitrary rule, like "the next animal that comes in after a case will be a control", and then you divide the cases and controls into exposed and unexposed groups. so, 1. select cases 2. select controls 3. divide by risk factor then, you can get an odds ratio OR = AD/CB. OR is found only in case-control study. The controls here are B and D - the non cases. You start with A + C first, then you get B + D, and then you divide A+C into A and C based on exposure, and so forth. cohort study - the only way you can measure incidence (cumulative incidence). if true incidence is small, ci = i. units of incidence are always per time! if you report ci, you must always report length of time during which observations were made. disadvantage is, if you are dealing with a rare disease, you have to collect a huge # of animals in the beginning to ensure that you get enough data. also, if induction time for dz is long, eg age of dz onset is large, you have to wait a long time for animals to get dz, so it's really expensive, b/c you have to keep monitoring the cohort. the longer it takes,the more risk of losing members of study - animals die for other reasons, people leave, farms go out of business. length of time, expense, and sheer magnitude make cohort studies less desirable. case control studies are becoming more prevalent. they do not measure incidence - they measure the odds ratio, which is the same as ie/io (AD isn't equal to ie, but AD/CB = ie/io). so if you have a small number of cases, because this is a rare disease, you can still do this study. you optimally only have to collect twice as many controls as cases to get a good answer. so you have fewer subjects, it's cheaper, less time consuming - you don't have to wait for disease to occur naturally. cheaper, fewer animals, more quickly completed - easier to get funding. retrospective cohort studies - use hospital records, reconstruct your cohort backwards. can't measure RR from data collected in a case control study. RR = cohort OR = case control but, OR is a good index of association, which is what you are testing, anyway. irrespective of how data are recorded and collected. often cohort studies are analyzed using an OR. OR is a Good Thing that can be applied across the board. **Bias** the forms of bias can mess up studies. bias is a systematic error. we always get random error - this is normal and part of doing observation studies. you can't get a group entirely free of random error - it tends to balance out. however, systematic error will *not* balance out. it will consistently skew the results. the most troubling bias is what is called **confounding** Confounding occurs when you think you are measuring the effects of one factor, when you are really measuring the effects of at least two factors. * factor 1 (drinking) <-- factor of interest * factor 2 (smoking) <-- confounder: has at least two properties: 1. it must affect incidence, and 2. it must be associated with factor 1. numeric example of confounding - made up example concerning bacterial infections and mastitis. there are many kinds of bacteria which cause mastitis in cattle. suppose we looked just at staph. see if infection with staph is associated with an increase in mastitis bacteria mastitis total staph + 100 1000 - 25 1000 make your 2x2 table: disease ci characteristic + - staph + 100 900 .1 staph - 25 975 .02 OR = AD/BC = 4.3 now, pretend this was a cohort study. what's the relative risk? RR = A/A+B so RR = 4 ------- C/C+D so, is it true that staph being there increases the risk of mastitis fourfold? maybe something else is associated with staphylococcal infections? say you find that animals that have staph also tend to have another bacteria more often than you would normally expect. bacteria mastitis total staph strep + + 80 667 + - 20 333 - + 10 250 - - 15 750 Now, we have to stratify our first analysis for the effect of strep. now we have to find two groups - one with strep, and one without strep. strep is our confounder. staph strep + + 80 667 - + 10 250 + - 20 333 - - 15 750 Now, compare those with staph to those without staph, using the RR. OR in strep + group = 3 OR in strep - group = 3 RR in strep + group = 2.7 RR in strep - group = 4.4 our RR was 4.3 before. this means that the effect of the confounder is to increase the incidence - when we take out the confounder, if RR is lower, then the effect of confounder is to increase disease. what is "interaction" ? normally in epidemiology we assume that different risk factors are additive in effect. if risk of dz due to exposure a is A, and risk due to b is B, then risk due to a and b together should be A + B. but, often it is actually greater than A+B. why? we account for this by saying that in addition to all the cause attributed to A and to B, there is some synergy b/w A and B. say smoking causes skin cancer because lip absorbs nicotine. say sun also causes lip cancer. well, if you get sunburnt lips and you smoke, maybe nicotine gets in faster - and this is an interaction that wouldn't have occurred if you were exposed to the factors separately. so when you stratify, if there is no interaction, your RRs should be very similar. sometimes, they are very different though. then there may be an interaction. ---end---