> qsr(
B
.Hannia Campos/0DTimes New RomanDbDbPbj0tbtb`0DArialNew RomanDbDbPbj0tbtb`0" DWingdingsRomanDbDbPbj0tbtb`0(
@n?" dd@ @@``rj5
"N5
S~1?0f@8g43d3db`0xbppp@<4!d!d4vDbg4KdKdb`0xbx p@pp<4BdBd4vDbRJ___PPT9,z?O=LValidity and Reproducibility of Exposure Measurements: The Method of TriadsMMGEdmond K. Kabagambe
and
Hannia Campos
Harvard School of Public Health:H,,!(0&%Suggested readingKaaks, RJ. (1997). Am J Clin Nutr 65:1232S1239S
Ocke and Kaaks (1997). Am J Clin Nutr 65:1240S1245S
Kabagambe et al (2001). Am J Epidemiol 154:11261135 $((b
, '
ObjectivesDefine the method of triads (MOT)
Describe the MOT and interpretation of the resulting validity coefficients
Give an example to illustrate the application of the MOT FPW((8((Exposure in EpidemiologyExposure can be
Discrete e.g. Yes or No
Continuous e.g. plasma cholesterol, lycopene intake
Measurement error is common with continuous exposuresNPMZ6P(M$6(
An example for illustration A researcher wants to measure dietary intake of lycopene
Three methods are available
Foodfrequency questionnaire
24hr dietary recalls/records (DR)
Biomarker (plasma concentrations)0UbU(b$
6We obtain 3 observed and 1 latent exposure parameters 75~FFQ estimate (Q)
Average of dietary recalls (R)
Biomarker measurement (M)
True, but unknown, exposure i.e. lycopene intake (T)*J,5, IRelation between true exposure (T) and 3 surrogate measures (Q, R, and M)JJValidity coefficients (1) <,How are they computed?
What do they measure?0+Validity coefficients (2) <]How are they interpreted for validity or reproducibility?
Can they have confidence intervals?^^0 ,Example on the validation of lycopene intakeiStep 1: Compute correlations
Step 2: Compute VCs
Step 3: Compare VC
Step 4: Compute 95% confidence limits&j1,9,.6Example on the validation of lycopene intake continued77iStep 1: Compute correlations
Step 2: Compute VCs
Step 3: Compare VC
Step 4: Compute 95% confidence limits&j1,9,0
ConclusionThe method of triads is a simple technique for assessing validity and reproducibility of continuous exposure measurements.
Although described for nutritional epidemiological studies, this method could also be applied to other continuous exposures.2P9(((/P&()* ,
/1
Pbc ` f33f` f̙f` ___` ff` 3ff3f>?" dd@,?lKZ@ dd lA@ d`" n?" dd@ @@``PRF @ ` `p>> L(
LT Qy!
LQy!"j
L
:B7 CDEHFPv@?q6 6 n(dx G
bq%(@B
y2
L
#BCE$HI`TQ1? `T`T`T`T`T`T`T`T`T`TQ!
L
N$vgֳgֳ ?Pv
T Click to edit Master title style!
!"<
L
Hv"ֳ"ֳ ?v
RClick to edit Master text styles
Second Level
Third Level
Fourth Level
Fifth Level!
S"
L
Tvgֳgֳ ?qv
A*"
L
TD vgֳgֳ ?. v
?*"
L
T vgg ?!gv
O*&"f
LNdh? ?L f33f Training ld` P(
PT *
P*"j
P
:BsC5
DEHFPv@?3
r4
rv$[I_l3HxF 3
%(@
2
P
#BCE$HI`TQ1? `Ta
`Ta
`T`T`Ta
`T`T*
P
NĴgֳgֳ ?/Ov
T Click to edit Master title style!
!"
P
Hgֳgֳ ?"pPv
W#Click to edit Master subtitle style$
$"
P
T䵐gֳgֳ ?qv
A*"
P
TDgֳgֳ ?0
v
?*"
P
Tgg ?!gv
A*"f
PNdh? ?P f33f0F>@D(
D
D
T$Y1 ?P
v
A*
"
D
TY1 ? v
C*
"r
D
c$ ?
v"H
D
TY1 ?
@v
RClick to edit Master text styles
Second level
Third level
Fourth level
Fifth level!
S"
D
ZDZ1 ?`P v
A*
"
D
ZZ1 ?` v
C*
"H
D0h ? ̙330<Y( 
<
<
T$v1 ?P
v
A*
"
<
T$%v1 ? v
C*
">
<
Z%v1 ?`P v
ZAssessing Validity and Reproducibility of Exposure Measurements using the method of triads[
["
<
Z%v1 ?` v
C*
"H
<0h ? ̙33f8p( )X
SP0P<$
0v
SdP0<$
0v
Z
Bdh ? f33f
$(
r
S$vLP
v
r
Sd$vLv
Z
Bdh ? f33f
0(
x
c$$LP
x
c$L
Z
Bdh ? f33f
8tL(
t
t c$DLP
"
t c$L
"B
ts*h ? f33f
8`L(
`
` c$ĺLP
"
` c$$L
"B
`s*h ? f33f
8L(

 c$DL
"
 c$L@`
"B
s*h ? f33f
q
i
4
(
r
SdL
U 8 j
4 ojxB
&
HD>?pPxB
'
HD>?p0xB
(
HD>?P0
)
ZĽ>?P
5T(xB
*
HD>?p@xB
,
HD>?PPP
xB

HD>?@0
.
Z$>?
5Q(
/
Z>?
5M(
0
Z>?z
5R(
1
Nd>?
P
arQM(((
2
N>?uZ
arQR(((
3
Z$>?
j
arRM(((B
s*h ? f33f
80@(
SLP
"
SDL
"B
s*h ? f33f
8PL(
c$dLP
"
c$L
"B
s*h ? f33f
8p\L(
\
\ c$LP
"
\ c$L
"B
\s*h ? f33f
8L(
c$LP
"
c$dL
"B
s*h ? f33f
8L(
c$$LP
"
c$L
"B
s*h ? f33f0<4T(
TX
T CD
4
T SķD
@
2Dr. Hannia Campos is an Associate Professor of Nutrition at Harvard School of Public Health, Boston, MA. The focus of her laboratory is to investigate whether genetic variation can modulate the effect of dietary intake on metabolic parameters that promote atherosclerosis and increase the risk of CHD. Such studies require assessment of the validity and reproducibility of the exposure in question.
Dr. Edmond K. Kabagambe is an epidemiologist and a postdoctoral fellow in the Department of Nutrition. He works with Dr. Campos on validation of dietary assessment methods and risk factors for myocardial infarction in the Costa Rican population.
Contact Information
Department of Nutrition, Building II, Room 353A655 Huntington AvenueBoston, MA 02115Email: hcampos@hsph.harvard.edu or ekabagam@hsph.harvard.edu
3aEaaaaaaaaXaaaaaAH
T0h ? ̙33
0dF(
d^
d SD
d c$D
@
<A researcher wants to measure the longterm intake of lycopene, a dietary antioxidant, and determine whether it is associated with decreased risk of heart disease. Three methods are available but vary in cost, accuracy and easy of application.
Foodfrequency questionnaire (FFQ or simply Q): This method is cheaper and easier to administer and is the most used method in nutritional epidemiological studies
Dietary recalls/records (R): May be more accurate than the FFQ but could be more expensive and require repeated measurements. These measurements could be correlated.
Biomarker (M): Here lycopene concentrations in plasma are measured. The method may be more accurate but is rather expensive and involving.
Because of the cost and time, two methods R and M are applied only on a subsample (e.g. 1020%) of the study subjects. The next slide shows data from a design with three methods for measuring the exposure.a0asaaaaNaH
d0h ? ̙3300x(
x^
x SD
x c$D
@
v
0 Measurement error is an important and common source of bias in epidemiological studies. This is particularly so when measurements are made with a laboratory test or an instrument (e.g. questionnaire) that is not considered to be a gold standard. Because of this potential for error, it is important to always validate the exposure measurements before detailed epidemiological analyses.
Assessment and detection of error is usually achieved through validity and reproducibility studies on a subsample of study subjects. In such studies, measurements by the gold standard or an alloyedgold standard (A) are compared with those from a field method (B) using correlation coefficients or by comparison of the means from the two methods. As we shall point out, these measures could be inaccurate if the two methods have correlated errors or if one of them has repeated measurements (which could also be correlated).
In the following slides we will (a) describe a technique, the method of triads , that is used to reduce this limitation and (b) apply this method using an example from nutritional epidemiology. We will first describe the example used to illustrate the method.5a"akaaaaaaFa4aaaa
aCaaaH
x0h ? ̙33R
0
(
^
SD
c$D
@
In epidemiological studies, there is always the true but sometimes unknown (latent) value of the exposure e.g. true lycopene intake. Interestingly, it can be shown using theory from factor analysis (see references) that, if linear relationships exist between the true exposure and measurements by the three methods, then the 3 measurements can be related to the true exposure using a mathematical formula. In the next slide we show a pictorial relationship between the three methods (Q, R and M) and the true exposure (T).d
aaaaaaH
0h ? ̙330P}(
X
CD
v
S[D
@v
9The articles listed on the slide describe the method of triads and give examples on the application of this method. Although, they are not a prerequisite for understanding this lecture, the reader may find it rewarding to read the articles first. They contain the theoretical background for the method of triads. *:.H
0h ? ̙3304(
^
SD
c$丐D
@
*`The method of triads is a technique for estimating the correlation between an observed measure of exposure and the true, but unknown, exposure value. The correlation in this case is referred to as the validity coefficient (VC). This technique was first applied by Dr. Rudolph Kaaks for use in validation studies in nutritional epidemiology. Application of the method of triads requires at least 3 measurements of the same exposure and assumes that all the 3 measurements are linearly related to the true exposure (T). First, we will briefly refer to continuous exposures and the associated measurement error.a#7E
\H
0h ? ̙33
0
t
(
X
CD
t
SD
@
In the diagram on the left side, the 3 measures of exposure are represented by the 3 corners of the triangle, hence the name method of triads and each of the surrogate measures is hypothesized to be linearly related to the true exposure (T). Also, pairwise Pearson correlations between the 3 methods can be computed and are abbreviated as rQR, rQM and rRM.
The correlation between T and each of the surrogate methods (Q or R or M) is referred to as the validity coefficient (VC). If all the 3 methods were relatively valid we would expect the correlations (VC) between each of the methods (Q, R, and M) and the true exposure to be about equal. This is the basis for the use of VCs in validating exposure measurements.
Use of the VC instead of a simple correlation is of advantage because the VC reduces the likelihood of over/underestimation of the true correlation, if two of the surrogate methods were correlated or if repeated measurements from any of the methods were correlated.}:c dJv>VyH
0h ? ̙33^
0
@ (
X
CD
SD
@
Validity coefficients (VCs) are computed from the following formulas derived using theory from factor analysis:
VCQT = "((rQR * rQM)/rRM);
VCRT = "((rQR * rRM)/rQM);
VCMT = "((rQM * rRM)/rQR).
VCQT, VCRT and VCMT are the validity coefficients between the true intake and the FFQ, recalls and the biomarker, respectively.
As pointed out before, VCs measure the correlation between true exposure (T) and the measurements from each surrogate method. Validity coefficients are expected to lie between 0 and 1 but in cases of small sample size or high variation in the pairwise correlations or when one of the correlations is negative, VCs can lie outside 0 and I boundary or may even be inestimable. This situation is referred to as a Heywood case (see references).
p m ~9z
AH
0h ? ̙330x`(
^
SD
r
c$$D
@
8The method with the highest validity coefficient (0 to 1) is considered to have measurements closer to the true, but unknown, exposure values and is therefore preferred. Because of the assumption that if all methods were valid they should give equal VCs, we can use these estimates to comment on which method performed better. The method with the highest VC is considered more valid!
In other cases one method (e.g. the field method) may be repeated (e.g. after a year) to give two values (e.g. Q1 and Q2). Then, as before, compute VCs first using Q1 alone and secondly using Q2. In a case where the method is reproducible, we would expect the VCs obtained using Q1 measurement to be equal to those obtained using Q2 measurements. This is the assessment of reproducibility using the method of triads.
Various methods are available for constructing confidence intervals about the VCs. One easy to execute method is the bootstrap method. Here samples of equal size are randomly drawn (with replacement) from the validation study sample for about 5001000 iterations. Validity coefficients are computed from each subsample resulting in a data set with 5001000 observations (each observation containing a VC for each method). The 5th and 95% percentiles can be computed and used as nonparametric confidence intervals (see references).9DgH)' SH
0h ? ̙33 0tl(
X
CD
l
SDD
@
In this example we have 4 measurements: Two from an FFQ administered twice (Q1 and Q2), an average of 7 repeated measurements from dietary recalls (R) and the measurement from plasma, a biomarker (M).
Questions: Are the 3 methods equally valid? Is the FFQ measurement fairly reproducible?
The pairwise correlations (roundedoff to 2 dec. points) are as follows:
For the FFQ and recalls: rQR = 0.37; or using Q2, rQR = 0.40;
For the FFQ and biomarker: rQM = 0.24 or using Q2, rQM = 0.18.
For the recalls and biomarker: rRM = 0.45.
The VCs (and bootstrap confidence intervals) follow:
VCQT = "((rQR * rQM)/rRM) = "((0.37 * 0.24)/0.45) = 0.44 (0.210.59);
VCRT = "((rQR * rRM)/rQM) = "((0.37 * 0.45)/0.24) = 0.84 (0.481.00)*;
VCMT = "((rQM * rRM)/rQR) = "((0.24 *0.45)/0.37) = 0.53 (0.350.90).
*Heywood case, upper 95% CL >1 has been set to 1.00.$U$Xe % ) 9 0 1 bR3$'I89bH
0h ? ̙33F0(
^
SD
c$D
@
vLooking at the VCs we can say that dietary recalls performed better than the FFQ and the biomarker and that all the three methods were relatively valid! Using the estimates from the second FFQ (Q2) we see that the VCs are about the same as when Q1 was used. Thus the FFQ is fairly reproducible.
The VCs (and bootstrap confidence intervals) when using Q2 follow:
VCQT = "((rQR * rQM)/rRM) = "((0.40 * 0.18)/0.45) = 0.40 (0.160.60);
VCRT = "((rQR * rRM)/rQM) = "((0.40 * 0.45)/0.18) = 0.98 (0.641.00);
VCMT = "((rQM * rRM)/rQR) = "((0.18 *0.45)/0.40) = 0.45 (0.230.73).
H<
G 0 0 .t88.H
0h ? ̙330D(
^
SD
c$D
@
:
H
0h ? ̙33ri8P ,h30$Z0ZTAPa@>Ng+Cs QE%:y<}Ō4R+VX1b
Oh+'0
px
0<H
T`hPowerPoint Presentationowe>D:\Program Files\Microsoft Office\Templates\1033\Training.pot 7 m F104Microsoft PowerPoint 7.0t O@PłL@@@ƻ~GoM ?6& &&#TNPP`0D
v
&
TNPP &&TNPP
&& "&&$<<$<xx<$xx$$,,$,hh,$hh$$$XX$XXj$T$?$HH+$HH$&&& "&$&8,z&&&
&&&&&&$<<$<xx<$xx$$,,$,hh,$hh$$$XX$XXj$T$?$HH+$HH$&
"&&&&9H@
 "Arial .12
LValidity and Reproducibility'!#!!$$*!$#$$!$ . f.12
JValidity and Reproducibility'!#!!$$*!$#$$!$ . .2
Lof Exposure Measurements:$'!#$!$ 0!! $!4!$!. f.2
Jof Exposure Measurements:$'!#$!$ 0!! $!4!$!. .%2
(LThe Method of Triads$$ 1 $#$#$!$ . f.%2
&JThe Method of Triads$$ 1 $#$#$!$ .yP  f 3.$2
}Edmond K. Kabagambe'$4$$$**!$!#!4$!. .2
and!$#."Arial 3.2
Hannia Campos*!$$!*!4$$!."Arial .62
o7Harvard School of Public Health'!# ! $ !&!."System&TNPP &$
՜.+,D՜.+,
:wrjTimes New RomanArial
Wingdings TrainingMValidity and Reproducibility of Exposure Measurements: The Method of TriadsSuggested readingObjectivesExposure in EpidemiologyAn example for illustration 7We obtain 3 observed and 1 latent exposure parameters JRelation between true exposure (T) and 3 surrogate measures (Q, R, and M)Validity coefficients (1) Validity coefficients (2)Example on the validation of lycopene intake7Example on the validation of lycopene intake continuedConclusion `(RZ
_PID_GUID_PID_HLINKSAN{38A763A131A411D69633DC03D676D367}A6http://www.hsph.harvard.edu/faculty/HanniaCampos.htmlRoot EntrydO)lp@Current User,SummaryInformation([0PowerPoint Document(
!"#$%&'()*+,./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\]^_`abcdfghijklw_㙵(C10Root EntrydO)!t@Current User(SummaryInformation([0PowerPoint Document(
!"#$%&'()*+,./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\]^_`abcdfghijklw _㙵
!"#$%&'()*+,./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\]^_`abcdfghijklnopqrstwRoot EntrydO)Current UsermSummaryInformation([0PowerPoint Document(DocumentSummaryInformation8e