Trial Under Fire: How drug testing lets us down
The parents of the little girl must have watched, horrified, as the days got darker and their daughter deteriorated. In December of 2000, she started taking the antidepressant Paxil.
During the months that followed the child experienced a Pandora’s box of side-effects: apathy, aggression, and insomnia. She developed skin discoloration and a personality disorder. On June 7, 2001, the girl had her last adverse reaction: suicide. Almost three years later, her death was reported to the database of the Canadian Adverse Drug Reaction Monitoring Program. The role of Paxil is listed as “suspected” and the outcome; “Died, drug may be contributory.” The girl was just 12 years old.
Report #167745 is one of several in Health Canada’s adverse drug reaction database concerning young people and Paxil, a top-selling antidepressant that was never officially licensed for children and teens but often prescribed for them. These kinds of drugs are seldom tested in children, so there is little data from large clinical trials, which scientists consider to be the “gold standard” in medical evidence. But pediatric patients still need medication, so clinicians might have to make do with what information is available and prescribe a drug as an unlicensed, or “off-label,” use.
But the case of Paxil is a bit different. There was more trial information available on the drug’s use in kids - it just wasn’t made public. In 2003, documents leaked to the media revealed that Paxil’s maker, GlaxoSmithKline, had evidence from two earlier trials showing the drug, part of a class called selective serotonin-reuptake inhibitors or SSRIs, did not benefit depressed children and teens. The trials had been done with the idea of expanding the drug’s market by applying to have it officially approved for use in pediatric depression.
The documents showed that, in 1998, the company intentionally withheld these negative results, knowing it could hurt its bottom line. As more trial evidence was revealed, it became apparent that the drug also could be harmful in some cases, doubling the rates of aggression and suicidal behaviour compared to patients taking placebo. It appeared Paxil did not help kids and for some it was highly dangerous.
Perhaps if the information about Paxil had been made available in 1998, the little girl in report # 167745 might have received another medication - or none at all.
After the media leak in 2003, Glaxo put the results of the trials in the public domain and information about the ineffectiveness of other newer antidepressants also came to light. By 2004, regulators in the United Kingdom and the U.S. had banned use of SSRIs in kids and Canada had issued a warning about them. “The disappointing reality is that antidepressant medications have minimal to no effectiveness in childhood depression beyond a placebo effect,” wrote Dr. Jane Garland, a professor of psychiatry at the University of British Columbia, in a 2004 editorial in the Canadian Medical Association Journal.
In the latest news, a comprehensive study done by scientists at the
U.S. Food and Drug Administration (FDA) showed that antidepressant
use also increases risk of suicidal thoughts and behaviour in young
adults. The team found that young adults aged 18 to 25 years who
took antidepressants were more than twice as likely to have suicidal
behaviour than those given sugar pills. The analysis, released in
December 2006, used pooled data from 372 antidepressant studies. It
included 11 antidepressants and involved almost 100,000
participants. Researchers found no such increased risk in older
adults using antidepressants.
One lesson emerges from the story of Paxil and pediatric depression:
drug company goals are not always the same as public health goals.
The incident also raises the question: should drug companies be
calling the shots when it comes to testing new drugs?
It’s comforting to imagine a team of Canadian wunderkinds huddled in
their publicly funded lab, comparing data and pouring over figures
to ensure our drugs are safe before they march onto pharmacy
shelves. But that’s a fantasy.
In the real world, a team of researchers handpicked by the drug
company usually tests our pharmaceuticals. The sponsoring company
funds the research, decides what questions about the drug need to be
answered and often decides whether the results should be made
public.
Since drug companies are in business to make a profit, they tend to
fund studies that explore benefits, not potential harms. “There’s no
rigorous study of drug interactions at all,” says Dr. Anne Holbrook,
director of clinical pharmacology at McMaster University in
Hamilton, Ont. “And yet if you ask physicians and patients what they
worry about most, drug interactions are always in their top five.”
It’s hard to imagine a comparable situation in any other industry
where human health and safety is at stake. Would we let airlines
decide how high their planes should be flown during safety tests? Or
allow them to test performance only when the aircraft is flying
through sunny skies?
Dr. Joel Lexchin, a Toronto emergency room physician and drug
industry critic, says the kind of safety problems we’ve seen with
blockbuster drugs in recent years can be traced back to the
limitations of the clinical trial.
“First of all, when drugs hit the market, they have only been tested
in maybe 5,000 or 6,000 people, which means that any adverse
reaction that occurs less than one in every 1,700 to 2,000 will not
have been seen in those trials,” says Dr. Lexchin, a professor in
York University’s School of Health Policy and Management. “Sometimes
you only find out safety issues when drugs are more widely
prescribed. And that’s one factor why you’ll have drugs being
withdrawn.”
Dr. Lexchin estimates that drug companies finance about 75% of all
clinical trials. Several studies have shown that trials sponsored by
drug companies are between two and five times more likely to show
their product in a favourable light, he adds.
One of the challenges with making trials relevant to the real world
is that drug companies tend to select “healthy” patients for their
studies. They want to show their drug is effective in as many people
as possible, so they include patients without multiple diseases and
who are not taking other medications. In some cases, this decision
might mean the drug will not be tested in the patients most likely
to use it. For example, Viagra was never tested in men with certain
heart conditions, even though these patients are among those most
likely to suffer from erectile dysfunction.
For years, researchers like Muhammad Mamdani have appealed for
clinical trials to be designed to apply to the patients in waiting
rooms.
“What happens when you get a homogenous group of people, you exclude
a ton of other people,” says Mamdani, a former senior scientist at
the Institute for Clinical Evaluative Sciences in Toronto who now
works for the drug company Pfizer in New York.
In a lot of trials it’s not rare to exclude over 80% of patients you
would normally see in practice. In many clinical trials that’s
fairly typical,” says Mamdani.
Trials are pricey. Rather than waiting years to complete one,
companies often choose to study drug effects over six to eight
weeks—shorter than the average course of treatment for many chronic
conditions. The researchers might also look at “surrogate
endpoints,” which can be seen faster, such as studying patient
cholesterol levels, rather than waiting to see if a drug actually
reduces the risk of heart attack or death.
Findings in these trials may also be skewed by the fact that
patients are meticulously monitored, with far more surveillance than
the real world allows. Mamdani suggests all these factors in the
testing phase - the homogenous patient groups, the limited time
frame, the surrogate endpoints and the extra monitoring - help set
the stage for the warnings and withdrawals we see with blockbuster
drugs.
Drug companies may also choose inappropriate comparators or less
than optimal doses to achieve favourable results. In general,
studies are not usually designed to look at safety - just whether it
works. They often don’t even have the numbers to try and examine
safety issues. And all you need to get a drug on the market in
Canada are positive results from two clinical trials.
“All you have to do to get a drug approved is have two positive
studies. And it doesn’t matter whether there are also six negative
studies, as long as you’ve got two positive studies, you can get the
drug approved,” says Dr. Lexchin. “But nobody ever knows about the
negative studies because chances that they’ll be published are very
small.”
If a drug company decides they want to try and expand the market for
one for their drugs, they are free to do the necessary studies and
then file an application. If the studies don’t turn out as they had
hoped, they could just drop the idea and never file the application.
The regulator might never see these negative results - as happened
in the case of Paxil and pediatric depression.
Adding fuel to the criticism of drug trials is the way that negative
results can be manipulated and obscured by industry. And even if
Health Canada is aware of negative results, the agency is legally
bound to keep the findings confidential.
Dr. Holbrook says that when conducting studies in-house, it’s easy
for industry to tinker with statistics to show good results, or
selectively report outcomes. Companies can simply “dredge the data”
until they come up with results that favour the product, she says.
This pattern is evident from viewing data disclosed by the FDA,
which is not bound by the same confidentiality clauses as Health
Canada, she adds.
While industry-led trials tend to produce favourable results,
there’s also evidence that researchers cherry-pick results from
their studies, often changing their game plan along the way. Dr. An-Wen
Chan and colleagues from Oxford University looked at 100 study
protocols and compared the published trials to see whether the main
outcomes changed. They found that more than 60% of the trials had
something changed, added or omitted by study’s end. For example, the
research team might have started out saying they were going to
report how the drug affected blood pressure, but changed their minds
along the way.
Dr. Chan, now a scientist with the University of Toronto, says his
team knew there was bias in the reporting of research, but didn’t
expect it would be so prevalent. “I think we were all quite shocked
to find the majority of studies actually altered the primary aim of
their study, the primary measure of whether their intervention
worked or not.”
Worse still, Dr. Chan showed in a follow-up study published in the
Canadian Medical Association Journal that even researchers receiving
public funds from the Canadian Institutes of Health Research (CIHR)
strayed from their original study blueprint.
Why are researchers doctoring results? It could be the “publish or
perish” pressures of academe. Medical journals are far more likely
to print positive findings than ones with negative ones or no
effect. Dr. Chan’s findings suggest that, if we are basing our
evidence-based medicine on data from these types of tidied-up
trials—we’ve got a problem.
In light of Dr. Chan’s study, the CIHR now requires all trials
funded by them to be registered and has become an international
cheerleader for full registration. Infamous cases like Dr. Nancy
Olivieri’s clash with the generic drug giant Apotex over the right
to disclose research findings have also had an impact. In the wake
of such well-known debacles, editors representing the world’s top
medical journals have taken steps to preserve their integrity.
These days, all researchers submitting papers for publication in
journals must show proof they retained control of their right to
publish data. The International Committee of Medical Journal Editors
has also demanded that all trial data submitted to them for
publication must have been registered at the outset, a move intended
to reduce the chances of data being omitted or altered by
researchers, whether in self-interest or at the behest of the drug’s
sponsor.
Critics fear that drug companies will resist full registration, but
industry insiders insist there is a willingness to increase
transparency. In September 2005, the International Federation of
Pharmaceutical Manufacturers and Associations launched a clinical
trials search engine to improve public access to information on
industry-funded trials. The portal links to online information about
ongoing and completed trials around the world and now indexes close
to 90,000 individual pages, according to the federation. It can be
reached at
www.canadapharma.org/ClinicalTrials/index_e.htmll
Participation in the portal is voluntary, but representatives of
Canada’s Research-Based Pharmaceutical Companies (Rx&D) say most
companies are on board. “We’ve come a long way on this issue,” says
Rav Kumar, vice-chair of the regulatory affairs committee for Rx&D.
Some observers remain skeptical about the new drive for
transparency. The drug industry is protective of their competitive
interests and has a track record of taking aim at those who question
their products. Canadian companies are no exception.
In 1997, the Canadian Coordinating Office for Health Technology
Assessment (CCHOTA) was slapped with a lawsuit by Bristol-Myers
Squibb in an attempt to block a scientific report on statins, which
the company said threatened sales of the drug Pravachol. CCHOTA, a
non-profit, publicly funded group, stood its ground and the drug
company eventually lost in court.
Dr. Holbrook also earned drug company ire when she was part of an
advisory group helping to draft treatment guidelines on a group of
ulcer drugs for the Ontario health ministry, which funded the
exercise. Industry often likes to fund these kinds of initiatives,
but this time the team declined to let pharmaceutical companies get
involved at all.
The group came up with guidelines for a class of popular ulcer drugs
called “proton pump inhibitors” and concluded that if used at a
proper dose, all the drugs could be used interchangeably. The
conclusion didn’t sit well with AstraZeneca, makers of Losec, a drug
that has brought them a hefty profit but was slightly more expensive
then a couple of the alternatives. Dr. Holbrook received a warning
letter from the company’s law firm. In the end, the letter delayed
publication of the guidelines, which, Dr. Holbrook notes, allowed
the company to keep reaping the rewards of the drug during that
time.
Registering is an obvious first step towards transparency and
optimizing information about our drugs, but researchers also want
changes to the ways trials are funded and designed.
Mamdani makes an appeal for large trials to be done with the real
world in mind, preferably before the drug hits the market. He noted
there were more than 200 trials looking at the class of COX-2
inhibitors such as Vioxx and most were virtually useless, focusing
on the same outcomes again and again. The money would have been
better spent on one, large pragmatic trial, he argues. “Those are
the trials that change practice.”
James McCormack has another solution. McCormack, a professor with
the Faculty of Pharmaceutical Sciences at the University of British
Columbia, says we should require all trials to answer a basic set of
questions, which could be determined by a group of experts. These
questions would address what patients really want to know about a
drug, such as whether it will reduce their chance of death from a
certain disease. Other areas could touch on adverse events and the
drug’s effect on quality of life - key factors that McCormack says
are almost never addressed.
The result would be consistent information about the endpoints that
are valuable to patients and clinicians and it would allow for
meaningful comparisons, he says.
Of course, the question of trial design is inextricably linked to
who pays for the trial. Some experts say that if Canada is going to
build a reliable drug safety system we must fund pre-market testing
- a daunting thought. According to Rx&D it costs an average of $1.3
billion to bring a new drug onto the market and takes an average of
12 years.
Others, like Arthur Schafer, say drug companies could continue to
fund the work, but pass on the funds to a third, independent party.
“It could be that we would want to make the drug companies pay for
the research, but pay the money to an independent body which would
assign the project based on peer review to a competent researcher or
a team of competent researchers, with no drug company involvement in
the selection process and that would sever the need to please drug
companies,” says Schafer, director of the University of Manitoba
Centre for Professional and Applied Ethics.
Schafer suggests we could also recoup the cost of trials through
taxation. A drug company would pay later to gain access to research
done by publicly funded scientists. He insists that the public is
already paying a huge price for corporate-funded research.
“We don’t pay it in taxes, we pay it in the loss of objective
truth-seeking science. If we want to increase knowledge in the
public interest, it’s going to have to be paid for with public tax
dollars. That doesn’t mean that most, or all of this expenditure
can’t be recaptured by the companies that will eventually
commercialize this research.”
Rav Kumar of Rx&D says it’s important to understand that trials are
already strictly regulated. The government is closely involved in
trials, and companies must report data to Health Canada throughout
the process at each stage. The checks and balances already exist; he
argues, right down to the ethics review boards, independent bodies
that must approve trials before they begin. Kumar also says that
having a third party managing the trial would not result in a
stronger system, since industry boasts experts who may have spent 10
or 15 years in their fields - and are the best ones to oversee these
trials.
Kumar says greater harmonization with the rest of the world would be
a step in the right direction, allowing regulators and industry to
draw more trial information from a bigger patient pool. “I think a
better approach is to come at it from a global basis and see where
Canada can fit into that, as opposed to having a Canadian
invented-only concept. At the end of the day, Canada represents a
pretty small percentage in terms of the global patient population,”
he says, adding that industry and regulators from around the world
should work together to improve global standards.
Some critics might be skeptical of such collaboration. They insist
that drug companies, as corporations, have different goals for
clinical trials than the public healthcare system. “It’s an economic
goal,” says Dr. Lexchin. “It’s the ability to market the drug and to
market it to as wide a population as possible, which is not
necessarily a public health goal.”
Schafer says the reformist measures that have been proposed, such as
registering trials, are potentially helpful, but aren’t enough. “If
our judges were paid by litigants and our police were paid by
individual firms…we wouldn’t get the kind of policing or judging we
want,” he says.” He who pays the piper calls the tune.” We still can
use drug company money, he says, but we need to set up a way to
avoid pitfalls of influence. “I just don’t see that a conflict of
interest that wouldn’t be tolerated in our judicial system should be
tolerated in science.”
Series Continues:
| The Painful Truth | Trial Under Fire |
Taking Care of Business |
Damage
Control | Message in a Bottle |