> (k
/0LDTimes New Roman(b(b4bj0XbXb`0@ @@``
@n?" dd@ @@```k7
!" #
$%&
'(*+,./01234567)S~1?̙@g46d6ddb`0\bx ppp@<4BdBd(b(bxh
?%O=# Are the results valid?3Was the validity of the included studies appraised?!!Are the results valid?Was the validity of the included studies appraised?
When conducting a metaanalysis the overall results are weighted by the size of the contributing studies. However further weighting may be given to the most rigorously conducted studies. This can be achieved by making a judgement on the validity of the studies. This may be qualitative (a comment about key characteristics of the study) or quantitative where each study is scored say out of 10, and the results of the study are then weighted according to the score achieved.""Give an example of the Oxford scoring system.#$Are the results valid?:Were the assessments of the included studies reproducible?$#:Were the assessments of the included studies reproducible?If the quality of the studies have been assessed then would the reader be able to understand the scoring system used and apply it to the papers?
Some metaanalyses may appraise the studies individually and give a score out of ten, but not make transparent why a paper scored badly  was it because there were a lot of drop outs from the study,or because the randomisation had been poorly done etc.%%Are the results valid?0Were the results consistent from study to study?&
Combinability
eFirst of all one should ask whether the results of separate trials can be meaningfully combined?
What criteria were used to decide that studies were similar enough to be combined??
Issues include study designs, types of patients and outcomes E.g. Can we combine RCTs with observational studies, or studies using cotrimoxazole with those using trimethoprim?$bb>
'&0Were the results consistent from study to study?BEach study is considered to be from a different population, the rate varies from study to study and differences can be because of experimental error, chance or differences in the populations. There are tests for homogeneity (or heterogeneity) which help decide the degree of caution in interpreting or pooling the results.('0Were the results consistent from study to study?In general differences in treatments are likely to differ in magnitude rather than direction and this can be displayed graphically to support or refute homogeneity between studies. Results are homogeneous if they reflect the same true effect.
References
Tests for homogeneity of effect in epidemiologic investigation. American Journal of Epidemiology 1977; 106: 1259
Metaanalysis in clinical research. Annals of Internal Medicine 1987; 107: 224334")(UAdd in some diagrams showing studies with heterogeneity and some that are homogeneousVV *)II. What are the results?^1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups. In a meta analysis this is usually presented as a diamond with the 95% confidence intervals grouped about a line indicating the null hypothesis.
A number of outcomes may be considered at once so more than one diamond may be presented.D.1.+A
Metaanalysis ,*II. What are the results?1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses? I.e. what are the results of the individual studies. Do some studies stand out as being different. Have the log odds ratios been calculated or has the meta analysis managed to pool original data in which case other sub group analyses may be possible  one of the advantages of pooling the original data.L..F
2,II. What are the results?j1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses?
Can you calculate the Number Needed to Treat (NNT) from the results presented? This necessitates an estimate of the absolute risk in the treated and control groups rather than simply having the odds ratio presented.L.=.e./II. What are the results?2. How precise are the results?& /0II. What are the results?pHow precise are the results?
A meta analysis should always present the 95% confidence intervals for each result.8q.038III. Will the results help me in caring for my patients?11. Can the results be applied to my patient care?22143Will the results help me in caring for my patients?1. Can the results be applied to my patient care? Clinical significance.
Refer back to the clinical problem
Are the studies generalisable to our patient?
Age, ethnicity, community or hospital patients etc?<I1{E2GeneralisabilityMajor problem with RCTs  i.e. do the results of this study apply to my patients?
Need to know details of patients included in all the studies. This will help decision about the generalisability of the results and also may reveal reasons for heterogeneity353Will the results help me in caring for my patients?82. Were all the clinically relevant outcomes considered?99463Will the results help me in caring for my patients?2. Were all the clinically relevant outcomes considered?
What about other outcomes  particularly harm.
What about quality of life issues?D9S9;573Will the results help me in caring for my patients?.3. Are the benefits worth the harms and costs?&/(683Will the results help me in caring for my patients?v3. Are the benefits worth the harms and costs?
Cost differences in treatments.
Greater benefits and less side effects?*/H/H73Critical Appraisal of a meta analysis  methodologyYSelection bias
Generalisability
Combinability
Consistency
Statistics
Sensitivity analysis8
StatisticscPool the log odds ratios
Standardised average
Regression analysis on pooled data
Sub group analysis9 Sensitivity analysisCan be done using quality score of trials e.g. published vs unpublished, observational and RCTs, details of randomisation
Publication bias  i.e. how many studies showing no difference would have to exist but not published to invalidate my results?,8/RSTUVWXYZ [!\"]$^%_&`'a(b)c*d+e,fg.h/i0j1k2y ` ̙33` 3` 3333f` 999MMM` f` f3` 3>?" dd@,?" dd@ " @ `" n?" dd@ @@``PR @ ` `p>>;( 5z
ZhEgֳgֳ ?P
T Click to edit Master title style!
!:
TgEgֳgֳ ?
RClick to edit Master text styles
Second level
Third level
Fourth level
Fifth level!
S
`tjEgֳgֳ ?``
=*
`jEgֳgֳ ?`
?*
`kEgֳgֳ ?`
?*H
0̙h? ? 999MMM $Blank Presentation0@0(
H
0hU? ? ̙33d4,0( d`
Ttғgֳgֳ ?PP
=*
TГgֳgֳ ? P
?*
Zғgֳgֳ ?@P` C
=*
Z֓gֳgֳ ?@ `C
?*H
0hU? ? ̙33
%$ ( X
f֓gֳgֳ ?P
fϓgֳgֳ ?
H
0h ? 999MMM
%$P(
fӓgֳgֳ ?P
C
fғgֳgֳ ?C
H
0h ? 999MMM
%$`(
f4͓gֳgֳ ?P
C
fדgֳgֳ ?C
H
0h ? 999MMM
%$p(
ftՓgֳgֳ ?P
C
fD.vgֳgֳ ?C
H
0h ? 999MMM
%$(
f.vgֳgֳ ?P
C
f2vgֳgֳ ?C
H
0h ? 999MMM
%$( D
fd2vgֳgֳ ?P
C
f0vgֳgֳ ?C
H
0h ? 999MMM
%$(
fvgֳgֳ ?P
C
f/vgֳgֳ ?C
H
0h ? 999MMM
%$(
f3vgֳgֳ ?P
C
fD4vgֳgֳ ?C
H
0h ? 999MMM
%$(
f6vgֳgֳ ?P
E
f6vgֳgֳ ?E
H
0h ? 999MMM
!%bZ(
f5vgֳgֳ ?P
E
H
0h ? 999MMM
"%$( X
$^
$
6?``^
$
6?`
$ fKxaxa ?P
E
$ ftKxaxa ?E
H
$0h ? 999MMMB
2%n(
#lLgֳgֳ ?
E
^
61?pB
H1?ppP
pB
H1?pp
NTMgֳgֳ?0`
<
PETO GRAPH(2
NNgֳgֳ?`@
BA
B
C
D
E
Total 2pB
H1?
pB
H1?Q
pp
NNgֳgֳ?`
= Outcome A
(2
N4Ogֳgֳ?
F (2
NOgֳgֳ?
10 2
NOgֳgֳ?0t
7Study 2^B
61?0P
NTPgֳgֳ?
;Placebo(2
NPgֳgֳ?0
8Drug(2^
61?` pB
H1?^
61?ppB
H1?
ZQgֳgֳ1?`
<pB
H1?^
61? 0
pB
H1? ^
61?
@@pB
H1?H
0h ? ̙33
$%4( z
4^
4
6?``^
4
6?`
4 ftQxaxa ?P
E
4 fQxaxa ? E
H
40h ? 999MMM
%%<(
<^
<
6?``^
<
6?`
< fRxaxa ?P
E
< fTSxaxa ?E
H
<0h ? 999MMM
&% D( 0tV
D^
D
6?``^
D
6?`
D fUxaxa ?P
E
D fUxaxa ?E
H
D0h ? 999MMM
'%0L( 0L
L^
L
6?``^
L
6?`
L fdkxaxa ?P
E
L fkxaxa ?E
H
L0h ? 999MMM
(%@T(
T^
T
6?``^
T
6?`
T flxaxa ?P
E
T fDmxaxa ?E
H
T0h ? 999MMM
)%P\(
\^
\
6?``^
\
6?`
\ fnxaxa ?P
\ fdnxaxa ?
H
\0h ? 999MMM
*%$`d(
d
d fogֳgֳ ?P
d fogֳgֳ ?
H
d0h ? 999MMM
+%pl(
l^
l
6?``^
l
6?`
l fqxaxa ?P
l f$rxaxa ?
H
l0h ? 999MMM
,%t(
t^
t
6?``^
t
6?`
t frxaxa ?P
t fDsxaxa ?
H
t0h ? 999MMM
%(
^

6?``^

6?`
 fdtxaxa ?P
 ftxaxa ?
H
0h ? 999MMM
.%( 0D
^
6?``^
6?`
f$uxaxa ?P
fuxaxa ?
H
0h ? 999MMM
/%$(
fDvgֳgֳ ?P
fvgֳgֳ ?
H
0h ? 999MMM
0%$(
fwgֳgֳ ?P
fgֳgֳ ?
H
0h ? 999MMM
1%$( 1
f$.gֳgֳ ?P
f.gֳgֳ ?
H
0h ? 999MMM0`( ֳ
d
c$1 ?Qv
T1gֳgֳ ?@v
H
0hU ? ̙33!0`( X
d
c$1 ?Qv
T0gֳgֳ ?@v
H
0hU ? ̙33"0`( X
d
c$1 ?Qv
T1gֳgֳ ?@v
H
0hU ? ̙33$0`(
d
c$1 ?Qv
TD2gֳgֳ ?@v
H
0hU ? ̙33#0 `( ֳ
d
c$1 ?Qv
T2gֳgֳ ?@v
H
0hU ? ̙33%00`( ֳ
d
c$1 ?Qv
T3gֳgֳ ?@v
H
0hU ? ̙330@`( ֳ
d
c$1 ?Qv
Td3gֳgֳ ?@v
H
0hU ? ̙33&0P`( ֳ
d
c$1 ?Qv
T3gֳgֳ ?@v
H
0hU ? ̙33'0``( ֳ
d
c$1 ?Qv
T$4gֳgֳ ?@
H
0hU ? ̙33(0p `( x[X8
d
c$1 ?Q
T4gֳgֳ ?@
H
0hU ? ̙33$)0(t( X
(
(
T4xaxa ?@
.j
(
s*1 ?Z
H
(0hU ? ̙33$*08t( X
8
8
T5xaxa ?@
.j
8
s*1 ?Z
H
80hU ? ̙33$,0@t( X
@
@
T6xaxa ?@
.j
@
s*1 ?Z
H
@0hU ? ̙33$/0Ht( X
H
H
Td6xaxa ?@
.j
H
s*1 ?Z
H
H0hU ? ̙33$00Pt(
P
P
T6xaxa ?@
.j
P
s*1 ?Z
H
P0hU ? ̙33$30Xt( X
X
X
T$7xaxa ?@
.j
X
s*1 ?Z
H
X0hU ? ̙33$40`t( X
`
`
T7xaxa ?@
.j
`
s*1 ?Z
H
`0hU ? ̙330h`( ֳ
hd
h
c$1 ?Q
h
T7gֳgֳ ?@
H
h0hU ? ̙33$50pt( X
p
p
TD8xaxa ?@
.j
p
s*1 ?Z
H
p0hU ? ̙33$60 xt( X
x
x
T8xaxa ?@
.j
x
s*1 ?Z
H
x0hU ? ̙33$700t( X
T9xaxa ?@
.j
s*1 ?Z
H
0hU ? ̙33$80@t(
Td9xaxa ?@
.j
s*1 ?Z
H
0hU ? ̙330P`(
d
c$1 ?Q
Tgֳgֳ ?@
H
0hU ? ̙330``(
d
c$1 ?Q
TDgֳgֳ ?@
H
0hU ? ̙33 0p`(
d
c$1 ?Q
Tgֳgֳ ?@
H
0hU ? ̙33dA0$(
Z1 ?V
3rD5gֳgֳ ?@
H
0hU ? ̙33r067' ;>AC
_PID_GUIDAN{771B2244DDBE11D69634008048154B39}
!
"#$
%&'(*+,./01234567)S~1?̙@g4!d!ddb`0\bx ~ppp@<4BdBd(b(bxh
?%O=# Are the results valid?3Was the validity of the included studies appraised?!!Are the results valid?Was the validity of the included studies appraised?
When conducting a metaanalysis the overall results are weighted by the size of the contributing studies. However further weighting may be given to the most rigorously conducted studies. This can be achieved by making a judgement on the validity of the studies. This may be qualitative (a comment about key characteristics of the study) or quantitative where each study is scored say out of 10, and the results of the study are then weighted according to the score achieved.""Give an example of the Oxford scoring system.#$Are the results valid?:Were the assessments of the included studies reproducible?$#:Were the assessments of the included studies reproducible?If the quality of the studies have been assessed then would the reader be able to understand the scoring system used and apply it to the papers?
Some metaanalyses may appraise the studies individually and give a score out of ten, but not make transparent why a paper scored badly  was it because there were a lot of drop outs from the study,or because the randomisation had been poorly done etc.%%Are the results valid?0Were the results consistent from study to study?&
Combinability
eFirst of all one should ask whether the results of separate trials can be meaningfully combined?
What criteria were used to decide that studies were similar enough to be combined??
Issues include study designs, types of patients and outcomes E.g. Can we combine RCTs with observational studies, or studies using cotrimoxazole with those using trimethoprim?$bb>
'&0Were the results consistent from study to study?BEach study is considered to be from a different population, the rate varies from study to study and differences can be because of experimental error, chance or differences in the populations. There are tests for homogeneity (or heterogeneity) which help decide the degree of caution in interpreting or pooling the results.('0Were the results consistent from study to study?In general differences in treatments are likely to differ in magnitude rather than direction and this can be displayed graphically to support or refute homogeneity between studies. Results are homogeneous if they reflect the same true effect.
References
Tests for homogeneity of effect in epidemiologic investigation. American Journal of Epidemiology 1977; 106: 1259
Metaanalysis in clinical research. Annals of Internal Medicine 1987; 107: 224334")(UAdd in some diagrams showing studies with heterogeneity and some that are homogeneousVV *)II. What are the results?^1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups. In a meta analysis this is usually presented as a diamond with the 95% confidence intervals grouped about a line indicating the null hypothesis.
A number of outcomes may be considered at once so more than one diamond may be presented.D.1.+A
Metaanalysis ,*II. What are the results?1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the
!" #
$%&
'(*+,./01234567)S~1?̙@g4!d!ddb`0\bx ~ppp@<4BdBd(b(bxh
?%O=# Are the results valid?3Was the validity of the included studies appraised?!!Are the results valid?Was the validity of the included studies appraised?
When conducting a metaanalysis the overall results are weighted by the size of the contributing studies. However further weighting may be given to the most rigorously conducted studies. This can be achieved by making a judgement on the validity of the studies. This may be qualitative (a comment about key characteristics of the study) or quantitative where each study is scored say out of 10, and the results of the study are then weighted according to the score achieved.""Give an example of the Oxford scoring system.#$Are the results valid?:Were the assessments of the included studies reproducible?$#:Were the assessments of the included studies reproducible?If the quality of the studies have been assessed then would the reader be able to understand the scoring system used and apply it to the papers?
Some metaanalyses may appraise the studies individually and give a score out of ten, but not make transparent why a paper scored badly  was it because there were a lot of drop outs from the study,or because the randomisation had been poorly done etc.%%Are the results valid?0Were the results consistent from study to study?&
Combinability
eFirst of all one should ask whether the results of separate trials can be meaningfully combined?
What criteria were used to decide that studies were similar enough to be combined??
Issues include study designs, types of patients and outcomes E.g. Can we combine RCTs with observational studies, or studies using cotrimoxazole with those using trimethoprim?$bb>
'&0Were the results consistent from study to study?BEach study is considered to be from a different population, the rate varies from study to study and differences can be because of experimental error, chance or differences in the populations. There are tests for homogeneity (or heterogeneity) which help decide the degree of caution in interpreting or pooling the results.('0Were the results consistent from study to study?In general differences in treatments are likely to differ in magnitude rather than direction and this can be displayed graphically to support or refute homogeneity between studies. Results are homogeneous if they reflect the same true effect.
References
Tests for homogeneity of effect in epidemiologic investigation. American Journal of Epidemiology 1977; 106: 1259
Metaanalysis in clinical research. Annals of Internal Medicine 1987; 107: 224334")(UAdd in some diagrams showing studies with heterogeneity and some that are homogeneousVV *)II. What are the results?^1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups. In a meta analysis this is usually presented as a diamond with the 95% confidence intervals grouped about a line indicaDocumentSummaryInformation8Root EntrydO)o q@Current User(SummaryInformation(,PowerPoint Document(11lh
!"#$%&'()*+,./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefgxijkmnopqrstuvwyz{}~
!"#$%&'()*+,./012345678 _
1b
Oh+'0
px
$0<
HT\No Slide TitleDr Ross LawrensoneC:\MSOffice\Templates\Blank Presentation.potowe ce\24Microsoft PowerPoint\Bl@PoR@pp@p/i
=@o qGoM C&ting the null hypothesis.
A number of outcomes may be considered at once so more than one diamond may be presented.D.1.+A
Metaanalysis ,*II. What are the results?1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses? I.e. what are the results of the individual studies. Do some studies stand out as being different. Have the log odds ratios been calculated or has the meta analysis managed to pool original data in which case other sub group analyses may be possible  one of the advantages of pooling the original data.L..F
2,II. What are the results?j1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses?
Can you calculate the Number Needed to Treat (NNT) from the results presented? This necessitates an estimate of the absolute risk in the treated and control groups rather than simply having the odds ratio presented.L.=.e./II. What are the results?2. How precise are the results?& /0II. What are the results?pHow precise are the results?
A meta analysis should always present the 95% confidence intervals for each result.8q.038III. Will the results help me in caring for my patients?11. Can the results be applied to my patient care?22143Will the results help me in caring for my patients?1. Can the results be applied to my patient care? Clinical significance.
Refer back to the clinical problem
Are the studies generalisable to our patient?
Age, ethnicity, community or hospital patients etc?<I1{E2GeneralisabilityMajor problem with RCTs  i.e. do the results of this study apply to my patients?
Need to know details of patients included in all the studies. This will help decision about the generalisability of the results and also may reveal reasons for heterogeneity353Will the results help me in caring for my patients?82. Were all the clinically relevant outcomes considered?99463Will the results help me in caring for my patients?2. Were all the clinically relevant outcomes considered?
What about other outcomes  particularly harm.
What about quality of life issues?D9S9;573Will the results help me in caring for my patients?.3. Are the benefits worth the harms and costs?&/(683Will the results help me in caring for my patients?v3. Are the benefits worth the harms and costs?
Cost differences in treatments.
Greater benefits and less side effects?*/H/H73Critical Appraisal of a meta analysis  methodologyYSelection bias
Generalisability
Combinability
Consistency
Statistics
Sensitivity analysis8
StatisticscPool the log odds ratios
Standardised average
Regression analysis on pooled data
Sub group analysis9 Sensitivity analysisCan be done using quality score of trials e.g. published vs unpublished, observational and RCTs, details of randomisation
Publication bias  i.e. how many studies showing no difference would have to exist but not published to invalidate my results?,8/RSTUVWXYZ [!\"]$^%_&`'a(b)c*d+e,fg.h/i0j1k2
%$P(
f.vgֳgֳ ?
E
f0vgֳgֳ ?E
H
0h ? 999MMMrP!!,kb/(k
/0LDTimes New Roman(b(b4bj0XbXb`0@ @@``
@n?" dd@ @@```k7 &&#TNPP`0D
v
&
TNPP &&TNPP
̙ " ! "&G&
 &Gy& y@H Times New Roman .(2
Are the results valid?*.yH Times New Roman̞ .2
R. .C2
v(Was the validity of the included studies(
. .2
1v
appraised?."System&TNPP &
՜.+,D՜.+, two groups.
What about subgroup analyses? I.e. what are the results of the individual studies. Do some studies stand out as being different. Have the log odds ratios been calculated or has the meta analysis managed to pool original data in which case other sub group analyses may be possible  one of the advantages of pooling the original data.L..F
2,II. What are the results?j1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses?
Can you calculate the Number Needed to Treat (NNT) from the results presented? This necessitates an estimate of the absolute risk in the treated and control groups rather than simply having the odds ratio presented.L.=.e./II. What are the results?2. How precise are the results?& /0II. What are the results?pHow precise are the results?
A meta analysis should always present the 95% confidence intervals for each result.8q.038III. Will the results help me in caring for my patients?11. Can the results be applied to my patient care?22143Will the results help me in caring for my patients?1. Can the results be applied to my patient care? Clinical significance.
Refer back to the clinical problem
Are the studies generalisable to our patient?
Age, ethnicity, community or hospital patients etc?<I1{E2GeneralisabilityMajor problem with RCTs  i.e. do the results of this study apply to my patients?
Need to know details of patients included in all the studies. This will help decision about the generalisability of the results and also may reveal reasons for heterogeneity353Will the results help me in caring for my patients?82. Were all the clinically relevant outcomes considered?99463Will the results help me in caring for my patients?2. Were all the clinically relevant outcomes considered?
What about other outcomes  particularly harm.
What about quality of life issues?D9S9;573Will the results help me in caring for my patients?.3. Are the benefits worth the harms and costs?&/(683Will the results help me in caring for my patients?v3. Are the benefits worth the harms and costs?
Cost differences in treatments.
Greater benefits and less side effects?*/H/H73Critical Appraisal of a meta analysis  methodologyYSelection bias
Generalisability
Combinability
Consistency
Statistics
Sensitivity analysis8
StatisticscPool the log odds ratios
Standardised average
Regression analysis on pooled data
Sub group analysis9 Sensitivity analysisCan be done using quality score of trials e.g. published vs unpublished, observational and RCTs, details of randomisation
Publication bias  i.e. how many studies showing no difference would have to exist but not published to invalidate my results?,8/RSTUVWXYZ [!\"]$^%_&`'a(b)c*d+e,fg.h/i0j1k2
%$(
fgֳgֳ ?
E
ftgֳgֳ ? @E
H
0h ? 999MMMrK$+#'.kb=(k
/0LDTimes New Roman(b(b4bj0XbXb`0@ @@``
@n?" dd@ @@```k7
!
"#$
%&'(*+,./01234567)S~1?̙@g4!d!ddb`0\bx ~ppp@<4BdBd(b(bxh
?%O=# Are the results valid?3Was the validity of the included studies appraised?!!Are the results valid?Was the validity of the included studies appraised?
When conducting a metaanalysis the overall results are weighted by the size of the contributing studies. However further weighting may be given to the most rigorously conducted studies. This can be achieved by making a judgement on the validity of the studies. This may be qualitative (a comment about key characteristics of the study) or quantitative where each study is scored say out of 10, and the results of the study are then weighted according to the score achieved.""Give an example of the Oxford scoring system.#$Are the results valid?:Were the assessments of the included studies reproducible?$#:Were the assessments of the included studies reproducible?If the quality of the studies have been assessed then would the reader be able to understand the scoring system used and apply it to the papers?
Some metaanalyses may appraise the studies individually and give a score out of ten, but not make transparent why a paper scored badly  was it because there were a lot of drop outs from the study,or because the randomisation had been poorly done etc.%%Are the results valid?0Were the results consistent from study to study?&
Combinability
eFirst of all one should ask whether the results of separate trials can be meaningfully combined?
What criteria were used to decide that studies were similar enough to be combined??
Issues include study designs, types of patients and outcomes E.g. Can we combine RCTs with observational studies, or studies using cotrimoxazole with those using trimethoprim?2bb$ >
'&0Were the results consistent from study to study?BEach study is considered to be from a different population, the rate varies from study to study and differences can be because of experimental error, chance or differences in the populations. There are tests for homogeneity (or heterogeneity) which help decide the degree of caution in interpreting or pooling the results.('0Were the results consistent from study to study?In general differences in treatments are likely to differ in magnitude rather than direction and this can be displayed graphically to support or refute homogeneity between studies. Results are homogeneous if they reflect the same true effect.
References
Tests for homogeneity of effect in epidemiologic investigation. American Journal of Epidemiology 1977; 106: 1259
Metaanalysis in clinical research. Annals of Internal Medicine 1987; 107: 224334")(UAdd in some diagrams showing studies with heterogeneity and some that are homogeneousVV *)II. What are the results?^1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups. In a meta analysis this is usually presented as a diamond with the 95% confidence intervals grouped about a line indicating the null hypothesis.
A number of outcomes may be considered at once so more than one diamond may be presented.D.1.+A
Metaanalysis ,*II. What are the results?1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses? I.e. what are the results of the individual studies. Do some studies stand out as being different. Have the log odds ratios been calculated or has the meta analysis managed to pool original data in which case other sub group analyses may be possible  one of the advantages of pooling the original data.L..F
2,II. What are the results?j1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses?
Can you calculate the Number Needed to Treat (NNT) from the results presented? This necessitates an estimate of the absolute risk in the treated and control groups rather than simply having the odds ratio presented.L.=.e./II. What are the results?2. How precise are the results?& /0II. What are the results?pHow precise are the results?
A meta analysis should always present the 95% confidence intervals for each result.8q.038III. Will the results help me in caring for my patients?11. Can the results be applied to my patient care?22143Will the results help me in caring for my patients?1. Can the results be applied to my patient care? Clinical significance.
Refer back to the clinical problem
Are the studies generalisable to our patient?
Age, ethnicity, community or hospital patients etc?<I1{E2GeneralisabilityMajor problem with RCTs  i.e. do the results of this study apply to my patients?
Need to know details of patients included in all the studies. This will help decision about the generalisability of the results and also may reveal reasons for heterogeneity353Will the results help me in caring for my patients?82. Were all the clinically relevant outcomes considered?99463Will the results help me in caring for my patients?2. Were all the clinically relevant outcomes considered?
What about other outcomes  particularly harm.
What about quality of life issues?D9S9;573Will the results help me in caring for my patients?.3. Are the benefits worth the harms and costs?&/(683Will the results help me in caring for my patients?v3. Are the benefits worth the harms and costs?
Cost differences in treatments.
Greater benefits and less side effects?*/H/H73Critical Appraisal of a meta analysis  methodologyYSelection bias
Generalisability
Combinability
Consistency
Statistics
Sensitivity analysis8
StatisticscPool the log odds ratios
Standardised average
Regression analysis on pooled data
Sub group analysis9 Sensitivity analysisCan be done using quality score of trials e.g. published vs unpublished, observational and RCTs, details of randomisation
Publication bias  i.e. how many studies showing no difference would have to exist but not published to invalidate my results?,8/RSTUVWXYZ [!\"]$^%_&`'a(b)c*d+e,fg.h/i0j1k2
%$(
fgֳgֳ ?0
E
fgֳgֳ ?
E
H
0h ? 999MMMrN.&[*.#^kb;(k
/0LDTimes New Roman(b(b4bj0XbXb`0@ @@``
@n?" dd@ @@```k7
!
"#$
%&'(*+,./01234567)S~1?̙@g4!d!ddb`0\bx ~ppp@<4BdBd(b(bxh
?%O=# Are the results valid?3Was the validity of the included studies appraised?!!Are the results valid?Was the validity of the included studies appraised?
When conducting a metaanalysis the overall results are weighted by the size of the contributing studies. However further weighting may be given to the most rigorously conducted studies. This can be achieved by making a judgement on the validity of the studies. This may be qualitative (a comment about key characteristics of the study) or quantitative where each study is scored say out of 10, and the results of the study are then weighted according to the score achieved.""Give an example of the Oxford scoring system.#$Are the results valid?:Were the assessments of the included studies reproducible?$#:Were the assessments of the included studies reproducible?If the quality of the studies have been assessed then would the reader be able to understand the scoring system used and apply it to the papers?
Some metaanalyses may appraise the studies individually and give a score out of ten, but not make transparent why a paper scored badly  was it because there were a lot of drop outs from the study,or because the randomisation had been poorly done etc.%%Are the results valid?0Were the results consistent from study to study?&
Combinability
eFirst of all one should ask whether the results of separate trials can be meaningfully combined?
What criteria were used to decide that studies were similar enough to be combined??
Issues include study designs, types of patients and outcomes E.g. Can we combine RCTs with observational studies, or studies using cotrimoxazole with those using trimethoprim?2bb$ >
'&0Were the results consistent from study to study?BEach study is considered to be from a different population, the rate varies from study to study and differences can be because of experimental error, chance or differences in the populations. There are tests for homogeneity (or heterogeneity) which help decide the degree of caution in interpreting or pooling the results.('0Were the results consistent from study to study?In general differences in treatments are likely to differ in magnitude rather than direction and this can be displayed graphically to support or refute homogeneity between studies. Results are homogeneous if they reflect the same true effect.
References
Tests for homogeneity of effect in epidemiologic investigation. American Journal of Epidemiology 1977; 106: 1259
Metaanalysis in clinical research. Annals of Internal Medicine 1987; 107: 224332")(UAdd in some diagrams showing studies with heterogeneity and some that are homogeneousVV *)II. What are the results?^1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups. In a meta analysis this is usually presented as a diamond with the 95% confidence intervals grouped about a line indicating the null hypothesis.
A number of outcomes may be considered at once so more than one diamond may be presented.D.1.+A
Metaanalysis ,*II. What are the results?1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses? I.e. what are the results of the individual studies. Do some studies stand out as being different. Have the log odds ratios been calculated or has the meta analysis managed to pool original data in which case other sub group analyses may be possible  one of the advantages of pooling the original data.L..F
2,II. What are the results?j1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses?
Can you calculate the Number Needed to Treat (NNT) from the results presented? This necessitates an estimate of the absolute risk in the treated and control groups rather than simply having the odds ratio presented.L.=.e./II. What are the results?2. How precise are the results?& /0II. What are the results?pHow precise are the results?
A meta analysis should always present the 95% confidence intervals for each result.8q.038III. Will the results help me in caring for my patients?11. Can the results be applied to my patient care?22143Will the results help me in caring for my patients?1. Can the results be applied to my patient care? Clinical significance.
Refer back to the clinical problem
Are the studies generalisable to our patient?
Age, ethnicity, community or hospital patients etc?<I1{E2GeneralisabilityMajor problem with RCTs  i.e. do the results of this study apply to my patients?
Need to know details of patients included in all the studies. This will help decision about the generalisability of the results and also may reveal reasons for heterogeneity353Will the results help me in caring for my patients?82. Were all the clinically relevant outcomes considered?99463Will the results help me in caring for my patients?2. Were all the clinically relevant outcomes considered?
What about other outcomes  particularly harm.
What about quality of life issues?D9S9;573Will the results help me in caring for my patients?.3. Are the benefits worth the harms and costs?&/(683Will the results help me in caring for my patients?v3. Are the benefits worth the harms and costs?
Cost differences in treatments.
Greater benefits and less side effects?*/H/H73Critical Appraisal of a meta analysis  methodologyYSelection bias
Generalisability
Combinability
Consistency
Statistics
Sensitivity analysis8
StatisticscPool the log odds ratios
Standardised average
Regression analysis on pooled data
Sub group analysis9 Sensitivity analysisCan be done using quality score of trials e.g. published vs unpublished, observational and RCTs, details of randomisation
Publication bias  i.e. how many studies showing no difference would have to exist but not published to invalidate my results?,8/RSTUVWXYZ [!\"]$^%_&`'a(b)c*d+e,fg.h/i0j1k2
%$(
fTgֳgֳ ?`
E
fgֳgֳ ?PE
H
0h ? 999MMMr_^(';^2kbC(k
/0LDTimes New Roman(b(b4bj0XbXb`0@ @@``
@n?" dd@ @@```k7
!
"#$
%&'(*+,./01234567)S~1?̙@g4!d!ddb`0\bx ~ppp@<4BdBd(b(bxh
?%O=# Are the results valid?3Was the validity of the included studies appraised?!!Are the results valid?Was the validity of the included studies appraised?
When conducting a metaanalysis the overall results are weighted by the size of the contributing studies. However further weighting may be given to the most rigorously conducted studies. This can be achieved by making a judgement on the validity of the studies. This may be qualitative (a comment about key characteristics of the study) or quantitative where each study is scored say out of 10, and the results of the study are then weighted according to the score achieved.""Give an example of the Oxford scoring system.#$Are the results valid?:Were the assessments of the included studies reproducible?$#:Were the assessments of the included studies reproducible?If the quality of the studies have been assessed then would the reader be able to understand the scoring system used and apply it to the papers?
Some metaanalyses may appraise the studies individually and give a score out of ten, but not make transparent why a paper scored badly  was it because there were a lot of drop outs from the study,or because the randomisation had been poorly done etc.%%Are the results valid?0Were the results consistent from study to study?&
Combinability
eFirst of all one should ask whether the results of separate trials can be meaningfully combined?
What criteria were used to decide that studies were similar enough to be combined??
Issues include study designs, types of patients and outcomes E.g. Can we combine RCTs with observational studies, or studies using cotrimoxazole with those using trimethoprim?2bb$ >
'&0Were the results consistent from study to study?BEach study is considered to be from a different population, the rate varies from study to study and differences can be because of experimental error, chance or differences in the populations. There are tests for homogeneity (or heterogeneity) which help decide the degree of caution in interpreting or pooling the results.('0Were the results consistent from study to study?In general differences in treatments are likely to differ in magnitude rather than direction and this can be displayed graphically to support or refute homogeneity between studies. Results are homogeneous if they reflect the same true effect.
References
Tests for homogeneity of effect in epidemiologic investigation. American Journal of Epidemiology 1977; 106: 1259
Metaanalysis in clinical research. Annals of Internal Medicine 1987; 107: 224332")(UAdd in some diagrams showing studies with heterogeneity and some that are homogeneousVV *)II. What are the results?^1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups. In a meta analysis this is usually presented as a diamond with the 95% confidence intervals grouped about a line indicating the null hypothesis.
A number of outcomes may be considered at once so more than one diamond may be presented.L.1.$ +A
Metaanalysis ,*II. What are the results?1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses? I.e. what are the results of the individual studies. Do some studies stand out as being different. Have the log odds ratios been calculated or has the meta analysis managed to pool original data in which case other sub group analyses may be possible  one of the advantages of pooling the original data.L..F
2,II. What are the results?j1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses?
Can you calculate the Number Needed to Treat (NNT) from the results presented? This necessitates an estimate of the absolute risk in the treated and control groups rather than simply having the odds ratio presented.L.=.e./II. What are the results?2. How precise are the results?& /0II. What are the results?pHow precise are the results?
A meta analysis should always present the 95% confidence intervals for each result.8q.038III. Will the results help me in caring for my patients?11. Can the results be applied to my patient care?22143Will the results help me in caring for my patients?1. Can the results be applied to my patient care? Clinical significance.
Refer back to the clinical problem
Are the studies generalisable to our patient?
Age, ethnicity, community or hospital patients etc?<I1{E2GeneralisabilityMajor problem with RCTs  i.e. do the results of this study apply to my patients?
Need to know details of patients included in all the studies. This will help decision about the generalisability of the results and also may reveal reasons for heterogeneity353Will the results help me in caring for my patients?82. Were all the clinically relevant outcomes considered?99463Will the results help me in caring for my patients?2. Were all the clinically relevant outcomes considered?
What about other outcomes  particularly harm.
What about quality of life issues?D9S9;573Will the results help me in caring for my patients?.3. Are the benefits worth the harms and costs?&/(683Will the results help me in caring for my patients?v3. Are the benefits worth the harms and costs?
Cost differences in treatments.
Greater benefits and less side effects?*/H/H73Critical Appraisal of a meta analysis  methodologyYSelection bias
Generalisability
Combinability
Consistency
Statistics
Sensitivity analysis8
StatisticscPool the log odds ratios
Standardised average
Regression analysis on pooled data
Sub group analysis9 Sensitivity analysisCan be done using quality score of trials e.g. published vs unpublished, observational and RCTs, details of randomisation
Publication bias  i.e. how many studies showing no difference would have to exist but not published to invalidate my results?,8/RSTUVWXYZ [!\"]$^%_&`'a(b)c*d+e,fg.h/i0j1k2
"%$( X
$^
$
6?``^
$
6?`
$ ftxaxa ?
$ fԞxaxa ?
H
$0h ? 999MMMB
2%n(
#lgֳgֳ ?p
^
61?`0pB
H1?ppP
pB
H1?pp
Ngֳgֳ?0`
<
PETO GRAPH(2
Ntgֳgֳ?`@
BA
B
C
D
E
Total 2pB
H1?
pB
H1?Q
pp
N4gֳgֳ?`
= Outcome A
(2
Ngֳgֳ?
F (2
Ngֳgֳ?
10 2
NTgֳgֳ?0t
7Study 2^B
61?0P
Ngֳgֳ?
;Placebo(2
Ngֳgֳ?0
8Drug(2^
61?` pB
H1?^
61?ppB
H1?
Ztgֳgֳ1?`
<pB
H1?^
61? 0
pB
H1? ^
61?
@@pB
H1?H
0h ? ̙33rn* IAJkbO(k
/0LDTimes New Roman(b(b4bj0XbXb`0@ @@``
@n?" dd@ @@```k7
!
"#$
%&'(*+,./01234567)S~1?̙@g4!d!ddb`0\bx ppp@<4BdBd(b(bxh
?%O=# Are the results valid?3Was the validity of the included studies appraised?!!Are the results valid?Was the validity of the included studies appraised?
When conducting a metaanalysis the overall results are weighted by the size of the contributing studies. However further weighting may be given to the most rigorously conducted studies. This can be achieved by making a judgement on the validity of the studies. This may be qualitative (a comment about key characteristics of the study) or quantitative where each study is scored say out of 10, and the results of the study are then weighted according to the score achieved.""Give an example of the Oxford scoring system.#$Are the results valid?:Were the assessments of the included studies reproducible?$#:Were the assessments of the included studies reproducible?If the quality of the studies have been assessed then would the reader be able to understand the scoring system used and apply it to the papers?
Some metaanalyses may appraise the studies individually and give a score out of ten, but not make transparent why a paper scored badly  was it because there were a lot of drop outs from the study,or because the randomisation had been poorly done etc.%%Are the results valid?0Were the results consistent from study to study?&
Combinability
eFirst of all one should ask whether the results of separate trials can be meaningfully combined?
What criteria were used to decide that studies were similar enough to be combined??
Issues include study designs, types of patients and outcomes E.g. Can we combine RCTs with observational studies, or studies using cotrimoxazole with those using trimethoprim?2bb$ >
'&0Were the results consistent from study to study?BEach study is considered to be from a different population, the rate varies from study to study and differences can be because of experimental error, chance or differences in the populations. There are tests for homogeneity (or hetero
!"#$%&geneity) which help decide the degree of caution in interpreting or pooling the results.('0Were the results consistent from study to study?In general differences in treatments are likely to differ in magnitude rather than direction and this can be displayed graphically to support or refute homogeneity between studies. Results are homogeneous if they reflect the same true effect.
References
Tests for homogeneity of effect in epidemiologic investigation. American Journal of Epidemiology 1977; 106: 1259
Metaanalysis in clinical research. Annals of Internal Medicine 1987; 107: 224332")(UAdd in some diagrams showing studies with heterogeneity and some that are homogeneousVV *)II. What are the results?^1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups. In a meta analysis this is usually presented as a diamond with the 95% confidence intervals grouped about a line indicating the null hypothesis.
A number of outcomes may be considered at once so more than one diamond may be presented.L.1.$ +A
Metaanalysis ,*II. What are the results?1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses? I.e. what are the results of the individual studies. Do some studies stand out as being different. Have the log odds ratios been calculated or has the meta analysis managed to pool original data in which case other sub group analyses may be possible  one of the advantages of pooling the original data.X..$F
2 ,II. What are the results?j1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses?
Can you calculate the Number Needed to Treat (NNT) from the results presented? This necessitates an estimate of the absolute risk in the treated and control groups rather than simply having the odds ratio presented.L.=.e./II. What are the results?2. How precise are the results?& /0II. What are the results?pHow precise are the results?
A meta analysis should always present the 95% confidence intervals for each result.8q.038III. Will the results help me in caring for my patients?11. Can the results be applied to my patient care?22143Will the results help me in caring for my patients?1. Can the results be applied to my patient care? Clinical significance.
Refer back to the clinical problem
Are the studies generalisable to our patient?
Age, ethnicity, community or hospital patients etc?<I1{E2GeneralisabilityMajor problem with RCTs  i.e. do the results of this study apply to my patients?
Need to know details of patients included in all the studies. This will help decision about the generalisability of the results and also may reveal reasons for heterogeneity353Will the results help me in caring for my patients?82. Were all the clinically relevant outcomes considered?99463Will the results help me in caring for my patients?2. Were all the clinically relevant outcomes considered?
What about other outcomes  particularly harm.
What about quality of life issues?D9S9;573Will the results help me in caring for my patients?.3. Are the benefits worth the harms and costs?&/(683Will the results help me in caring for my patients?v3. Are the benefits worth the harms and costs?
Cost differences in treatments.
Greater benefits and less side effects?*/H/H73Critical Appraisal of a meta analysis  methodologyYSelection bias
Generalisability
Combinability
Consistency
Statistics
Sensitivity analysis8
StatisticscPool the log odds ratios
Standardised average
Regression analysis on pooled data
Sub group analysis9 Sensitivity analysisCan be done using quality score of trials e.g. published vs unpublished, observational and RCTs, details of randomisation
Publication bias  i.e. how many studies showing no difference would have to exist but not published to invalidate my results?,8/RSTUVWXYZ [!\"]$^%_&`'a(b)c*d+e,fg.h/i0j1k2
$% 4( z
4^
4
6?``^
4
6?`
4 fԤxaxa ?0
4 f4xaxa ?00P
H
40h ? 999MMMr,**kbg(k
/0LDTimes New Roman(b(b4bj0XbXb`0@ @@``
@n?" dd@ @@```k7
!
"#$
%&'(*+,./01234567)S~1?̙@g4!d!ddb`0\bx ~ppp@<4BdBd(b(bxh
?%O=# Are the results valid?3Was the validity of the included studies appraised?!!Are the results valid?Was the validity of the included studies appraised?
When conducting a metaanalysis the overall results are weighted by the size of the contributing studies. However further weighting may be given to the most rigorously conducted studies. This can be achieved by making a judgement on the validity of the studies. This may be qualitative (a comment about key characteristics of the study) or quantitative where each study is scored say out of 10, and the results of the study are then weighted according to the score achieved.""Give an example of the Oxford scoring system.#$Are the results valid?:Were the assessments of the included studies reproducible?$#:Were the assessments of the included studies reproducible?If the quality of the studies have been assessed then would the reader be able to understand the scoring system used and apply it to the papers?
Some metaanalyses may appraise the studies individually and give a score out of ten, but not make transparent why a paper scored badly  was it because there were a lot of drop outs from the study,or because the randomisation had been poorly done etc.%%Are the results valid?0Were the results consistent from study to study?&
Combinability
eFirst of all one should ask whether the results of separate trials can be meaningfully combined?
What criteria were used to decide that studies were similar enough to be combined??
Issues include study designs, types of patients and outcomes E.g. Can we combine RCTs with observational studies, or studies using cotrimoxazole with those using trimethoprim?2bb$ >
'&0Were the results consistent from study to study?BEach study is considered to be from a different population, the rate varies from study to study and differences can be because of experimental error, chance or differences in the populations. There are tests for homogeneity (or heterogeneity) which help decide the degree of caution in interpreting or pooling the results.('0Were the results consistent from study to study?In general differences in treatments are likely to differ in magnitude rather than direction and this can be displayed graphically to support or refute homogeneity between studies. Results are homogeneous if they reflect the same true effect.
References
Tests for homogeneity of effect in epidemiologic investigation. American Journal of Epidemiology 1977; 106: 1259
Metaanalysis in clinical research. Annals of Internal Medicine 1987; 107: 224332")(UAdd in some diagrams showing studies with heterogeneity and some that are homogeneousVV *)II. What are the results?^1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups. In a meta analysis this is usually presented as a diamond with the 95% confidence intervals grouped about a line indicating the null hypothesis.
A number of outcomes may be considered at once so more than one diamond may be presented.L.1.$ +A
Metaanalysis ,*II. What are the results?1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses? I.e. what are the results of the individual studies. Do some studies stand out as being different. Have the log odds ratios been calculated or has the meta analysis managed to pool original data in which case other sub group analyses may be possible  one of the advantages of pooling the original data.X..$F
2 ,II. What are the results?j1. What are the overall results of the study?
Look at the Relative Risk (RR) of the main outcome in the two groups.
What about subgroup analyses?
Can you calculate the Number Needed to Treat (NNT) from the results presented? This necessitates an estimate of the absolute risk in the treated and control groups rather than simply having the odds ratio presented.d.=.$e ./II. What are the results?2. How precise are the results?& /0II. What are the results?pHow precise are the results?
A meta analysis should always present the 95% confidence intervals for each result.8q.038III. Will the results help me in caring for my patients?11. Can the results be applied to my patient care?22143Will the results help me in caring for my patients?1. Can the results be applied to my patient care? Clinical significance.
Refer back to the clinical problem
Are the studies generalisable to our patient?
Age, ethnicity, community or hospital patients etc?<I1{E2GeneralisabilityMajor problem with RCTs  i.e. do the results of this study apply to my patients?
Need to know details of patients included in all the studies. This will help decision about the generalisability of the results and also may reveal reasons for heterogeneity353Will the results help me in caring for my patients?82. Were all the clinically relevant outcomes considered?99463Will the results help me in caring for my patients?2. Were all the clinically relevant outcomes considered?
What about other outcomes  particularly harm.
What about quality of life issues?D9S9;573Will the results help me in caring for my patients?.3. Are the benefits worth the harms and costs?&/(683Will the results help me in caring for my patients?v3. Are the benefits worth the harms and costs?
Cost differences in treatments.
Greater benefits and less side effects?*/H/H73Critical Appraisal of a meta analysis  methodologyYSelection bias
Generalisability
Combinability
Consistency
Statistics
Sensitivity analysis8
StatisticscPool the log odds ratios
Standardised average
Regression analysis on pooled data
Sub group analysis9 Sensitivity analysisCan be done using quality score of trials e.g. published vs unpublished, observational and RCTs, details of randomisation
Publication bias  i.e. how many studies showing no difference would have to exist but not published to invalidate my results?,8/RSTUVWXYZ [!\"]$^%_&`'a(b)c*d+e,fg.h/i0j1k2
%%0<(
<^
<
6?``^
<
6?`
< fTxaxa ?0
< fxaxa ?
H
<0h ? 999MMMre,0kb