When is a Signal Not a Signal?
Let us take a look at some situations in signal detection where the issues are not crystal clear.
A Quick Review of Signals
First the definition. In fact, there are several definitions of signals floating around and all are fairly good. From the EMA’s GVP Module IX on signal management updated in 2017:
“…information arising from one or multiple sources, including observations and experiments, which suggests a new potentially causal association, or a new aspect of a known association between an intervention and an event or set of related events, either adverse or beneficial, which is judged to be of sufficient likelihood to justify verificatory action.”
From the Uppsala Monitoring Centre:
“Reported information on a possible causal relationship between an adverse event and a drug, the relationship being unknown or incompletely documented previously. Usually more than a single report is required to generate a signal, depending upon the seriousness of the event and the quality of the information.”
“This describes the first alert of a problem with a drug. By its nature a signal cannot be regarded as definitive but indicates the need for further inquiry or action. On the other hand it is prudent to avoid a multiplicity of signals based on single case reports since follow up of all such would be impractical and time consuming. The definition allows for some flexibility in approach to a signal based on the characteristics of individual problems. Some would like a “signal” to include new information on positive drug effects, but this is outside the scope of a drug safety Programme.” (Delamothe, Br Med J 1992;304:465)
Note that this applies to both marketed drugs and drugs still in clinical trials. There are differences, of course, but fundamentally the concepts are the same.
So what does this boil down to? Basically for drug signals that PV professionals work with we have this:
- New information that is bad, problematic and troublesome
- This information is hopefully reliable but not necessarily
- It is new or adds to previous (weak) suspicions of a problem
- It is not definitive and not necessarily, at first blush, due to the drug
- It is associated, perhaps only temporally, with the use of the drug in question
- It is usually more than one report but can be based on a single report in some cases
- It makes some people in the company unhappy and they will try to minimize or even deny it is due to the drug
- It may need further investigation to come to a conclusion on its meaning and what, if anything, should be done.
Signals may be specific or general. For example, here is a listing from FDA’s Serious Signal Website: medication error, neuropsychiatric AEs, pediatric fever and dehydration, musculoskeletal and connective tissue pain and discomfort, acute kidney injury and interstitial nephritis, injection site infection, liver dysfunction, interstitial pneumonia, vitiligo, decline in renal function, stomach polyps, ischemic colitis, Stevens Johnson/TEN. Note that some are clearly less urgent than others and some are quite non-specific.
Some signals are “derived” such as from increased frequency reporting which requires active work by the company to look at frequencies over time.
What to Do When Such Information Arrives
At this point, the company should already have in place a well-defined SOP on how to handle signals. This SOP should define a signal and note who is responsible (by job/position not by name) for determining that it is a (weak, possible, strong) signal and what is to be done to handle it.
If the issue is urgent and must be acted on immediately (e.g. tampering, counterfeits, some fatal or life-threatening SAEs etc.), then those steps delineated in the SOP should be put into place. If the issue is to be presented to someone else or a committee in the company, that should be done or scheduled depending upon the priority of the case.
Unless the case is complete or follow-up cannot be done, the missing medical information should be sought – urgently if this is a high priority case.
If the issue is critical whereby drug withdrawal, study stopping or other urgent action must occur, the SOP-defined action group must handle this immediately.
For the vast majority of signals, however, the issue does not require immediate action. Follow up information and outcome will be sought and presented expeditiously to the signaling team to determine what other measures might be needed such as further testing of the involved patients, label or protocol/consent changes etc.
Keep in mind that not all signals are dramatic or require immediate action. Many may be non-serious. As is well known, the safety profile of a drug is never well known at launch or during the clinical trials. So it is to be expected that new AEs, SAEs and signals will be picked up over time and added to the labeling.
So When is a Signal Not a Signal?
The first answer is that any safety issue, even if well known, may be a signal.
- A single report or two does not tell us about frequency over time and whether it is increasing. FDA has stated in multiple places that single SAEs are often not interpretable in terms of causality. In the December 2015 draft guidance on IND reporting FDA notes: “the types of serious adverse events that are interpretable based on single or small numbers of events. Some examples include angioedema, hepatic injury, Stevens-Johnson Syndrome, tendon rupture, agranulocytosis, and acute liver failure. Most serious adverse events, however, will not be readily interpretable as single events.” This is in the context of expedited (7/15 day) reporting. Thus a single SAE may not be expeditable but may indeed be a signal that one must keep an eye on with continued surveillance to see if it might indeed be drug related. The reverse is also true: a single case may be expeditable but not necessarily a signal.
- An AE/SAE may be in the label but may be due to a different reason. For example, suppose diarrhea is labeled and known. If there is, say, a manufacturing problem or tampering or change in excipients, the diarrhea may now be due not to the pharmacologic action of the drug but to one of the other reasons noted. Again, the incidence may be up but a single report won’t show this.
- Comment: So the lesson here is that any SAE or AE may be a signal even if labeled. That is why the signaling SOP should have a periodic review of all SAEs and AEs whether manually or programmatically or both looking at the aggregate and individual cases as needed. Medical judgment is needed.
The second answer depends on what is labeled in the reference safety information. This has become a complex and very problematic issue these days as there are now many different reference safety documents around the world. In theory, all labeling should be the same and AEs/SAEs and safety issues should be in all documents worldwide. That is not, however, always the case.
If a drug is not adequately and completely labeled in each country as required by that country, then the missing information should be considered a signal there if it has not yet reached the threshold for inclusion in the labeling. Failure to list safety issues can lead to patient safety issues and even death. It may also lead to lawsuits especially in the US.
The third and last answer is that certain very bizarre cases, outliers and other strange things should not be rejected out of hand a possible signal. For example, if a drug does not produce changes in the mean and median values of liver enzymes (e.g. AST, ALT) but a couple of patients had enzyme values 50 times normal, one cannot exclude this as a signal simply because the vast majority of patients had no problem. One must carefully study these patients to see if there is something different about them. For example, were they taking a new drug that might produce a drug-drug interaction not seen before. In fact, in the world of drug safety, outliers and overdoses are very important and interesting cases to study because they may reveal critical safety information.
So the bottom line is that a signal is not a signal only if one can clearly find evidence that strongly strongly suggests that the safety issue was due to something else. This is not that uncommon, in fact. A single case or a few cases which are weak (e.g. no negative rechallenges) may not be confirmed as more and more marketing data comes in. If additional cases do not occur after hundreds of thousands or millions or more patients use the drug, the signal will remain unconfirmed and may even be removed from the signal list.
So, sorry, but signals are signals and rarely, if ever, are signals not signals.
Tags: clinical trials, drug safety, Pharmacovigilance, signal detection, signals