|
Treatment, n |
Guess, n |
||
|
Experimental |
Control |
Don’t know |
|
|
Experimental |
|
|
|
|
Control |
|
|
|
Aims
To
develop web contents for www.blindingindex.org, where clinical trialists
used blinding can deposit the data of their studies on the success of
blinding.
Background
Why we blind?
Knowledge
about treatment allocation and its related expectation and behavior could
influence study outcomes in randomized clinical trials. Blinding (or masking)
aims to eliminate or minimize this. The
importance of blinding has been repeatedly emphasized by regulatory and
advisory agencies for clinical trials such as FDA and CONSORT group.1,2
Although blinding, (particularly subject- or therapist- blinding) is not
feasible in all disciplines of medicine, studies without blinding are
regarded to have inferior quality. While there may be multiple parties to be
blinded in trial such as subject, therapist, assessor, and data analyst.
Turning a
blind eye to blinding
Despite
good effort to implement
placebo (sham) control, there is very little evidence that blinding is
maintained throughout most trials. In response to this awareness, some clinical
trialists reported the results of guess of trial participants regarding the
group allocation they were in at the end of treatment or during the early stage
of treatments. However, this approach has largely been ignored.3,4
Literature reviews of blinding in clinical trials also called for urgent
improvement in this issue.5-8 Blinding-related data were presented
in different formats in publication, yet systematic approach to this matter is
not commonly adopted, because of the absence of consensus and limited
methodological research.
Guessing group
allocation at the end of treatment
Sporadically,
the data on the “guess” of study participants and those involved in trials revealed
that trial participants and staff correctly guessed their group allocation more
than can be explained by chance. This indicates that these studies might have
failed to achieve blinding. The interpretation of this potential unblinding is
an unanswered question.
When
participants should be asked for their guesses is another conundrum. Asking before and/or during the early stage
of a trial might give important information on participants' perception on the
intervention groups, whereas asking at the end of treatment would reflect
‘totalistic judgment’ of the interventions including their effects.9-12
Theoretically, more frequent information on the participants' guesses may
elucidate the detailed dynamics of the participants' judgment and belief during
the trial and provide important data on this issue. However, frequent
questioning of the participants may trigger their curiosity about their
treatment group. This may affect their whole attitude towards the study,
influencing factors such as non-compliance and drop-out that could make the
treatment effects observed questionable.13 This can jeopardize the
internal validity of the trial, and its practicality.
Asking for a
guess at the end of treatment will not affect the trial intervention and
practicality itself. However, this doesn't guarantee perfect assessment of blinding; the
group allocation may have been already unblinded by other factors including
therapeutic or adverse effects. By definition, in order for a trial to be
blinded, the treatment allocation needs to remain unknown during the entire
course of the trial or until the data have been collected.12,14
Objective methods to quantify the guess data and to determine the optimal time
for blinding surveys are in great need. In the meantime, it may be sensible to
conduct such surveys at the end of study.
Blinding Indexes
To deal with
data from these surveys on treatment guessing, James et al.15 and
Bang et al.16 developed blinding indexes (BIs). James et al.’s
method is a variant of the Kappa statistic, which focuses on disagreement
between treatment allocation and guess. Bang et al.’s BI estimates the
percentage of correct guessing beyond the level expected by chance in each
treatment group. A unique feature in blinding assessment is how to deal with
‘Don’t know’ answer. Since their
introduction, both BIs have been used in several studies to quantitatively
measure the level of correct guess, a surrogate indicator of unblinding. These
two BIs are based on different paradigms and assumptions, and, naturally, they
both have their own strengths and weaknesses.17, 20
In our recent
editorial,19 we adopted Bang et al.’s Blinding Index (BI) that
evaluates the blinding status in each group separately, enabling us to examine
different blinding patterns in different arms (i.e. Placebo group and treatment
group).17. Bang et al.'s approach allows us to distinguish nine
possible scenarios, determined by the combination of three possible results for
each of the two arms. The three possible results for each arm are “random”
(indicated by values of BI sufficiently close to 0), "unblinded"
(indicated by positive BI value, meaning that significant portion of participants guessed their
true allocation), and "opposite" (indicated by negative BI values,
meaning that significant portion of participants guessed the opposite treatment). From our
systematic review of 63 randomized placebo (sham) controlled trials, we
classified each trial into 9 (3*3) blinding scenarios.
Most studies unblinded
The most
compelling result visible from this table is that about 73.2% (46 out of 63)
studies turned out unblinded, mostly in the experimental group19.
Each scenario may have different stories, underlying causes and
interpretations. While further discussion of the meaning of this finding is a
topic of an independent study, the optimistic assumption that blinding remains
intact as intended throughout the trial proved to be untrue. This raises
several new questions: 1) whether success of blinding should be modeled as a
binary variable or as a scale variable; 2) what should be the critical cut
off point for BI in order to declare that blinding was successful; 3) which of
the above scenarios are desirable to attain satisfactory internal validity; 4)
can BI data evaluate the comparability between the groups; 5)what
are the impacts of blinding on the study results (including conclusions on the
treatment effectiveness/efficacy); and 6) how to understand placebo effect,
resurrecting the old question ‘Does it have power?’ The last question is
particularly relevant to the ‘wishful thinking’ scenario in the table.
Answering these questions is more complicated than we may expect, for which
much more data and researches are necessary.
Urging to report the BI
data
In the middle
of more questions than answers, a promising news is that a user-written command
for Stata to calculate the two BIs was released in March 2008.18
With this computational tool available, we hope to see more quantitative data
and evidence on blinding in the literature, and we hope to see more researches
studying the effect of blinding status on the outcomes. We urge all trials
claiming “blind” to report the BI at the end of trial and make some efforts to
understand qualitative as well as quantitative aspects of (un)blinding and
share their stories in research communities, which will shed some light on the
internal validity and the quality of trials, providing the communities with new
elements to interpret their findings and to plan good clinical trials in the
future.
The
need for a centralized BI data depository
With anticipated availability of more BI data, it would
be sensible to make a centralized web based data depository where more such
data can be gathered. BI data may be deposited by its managers as they are
identified, or trialists provide the website with either reference information
or BI data. This data pool will enable series of studies to look at the
association of blinding and other outcomes to be feasible. Providing blinding
data is absolutely voluntary.
References