Hello again, it's Janet. And I'm going to be talking about how the choice of outcomes may influence the design of a trial. So I wanted to start talking about efficacy versus effectiveness trials. So efficacy trials are usually those that are looking to be able to evaluate the condition under the best of all circumstances. Does this intervention change outcomes given sort of every benefit, versus effectiveness, which is looking how the intervention will work in a more real world setting. So you can imagine that this could reflect the choice of outcome. So, for just two examples, in a vaccine trial, if you were looking at effectiveness you might go to a larger population and look for all clinical cases of influenza. And may or may not be confirmed by laboratory evaluations. In a efficacy evaluation, if you were trying to evaluate whether this vaccine seem to effect and promote immunity, you might base the first studies on laboratory confirmation of antibody level. So really quite different outcomes and you'll need quite different sample size. Maybe in a more clinical realm if you were looking at asthma trial an effectiveness trial might look at actual things that happen to patients. Did you decrease hospitalizations or oral steroid courses? In an efficacy trial where you' were trying to see, well, does this treatment have a positive effect on some of the clinical measures of disease, you might look at FEV1, which is forced expiratory volume in one second. But a spirometric measure, and as I said before, you know, you can be walking around with a pretty poor FEV1 and not be hospitalized. So, you probably need a smaller sample size because of the type of outcome. The choice of the outcome is, is crucial to so many aspects of a clinical's trials design. It's usually based on prior research and also, you have to look at prior research to make sure that your study, with your results and your outcome, will be able to be integrated into the body of evidence about a treatment. It's important to know what other people measured to assure that your study can be evaluated in a larger context. Not necessarily mean that you have to copy people, but perhaps you may add another secondary outcome that reflects what others have used but may chose another primary outcome, that you may think is a better choice, for a variety of reasons. But it is a key factor for determining what the eligibility criteria are for a trial, the sample size, how long are you going to need to follow people to get changes in this outcome, or to have events occur? And what's the frequency? Do you need to see people frequently so you don't miss things, or is it something that people would remember happened in terms of events like an MI that, you know, maybe you don't have to follow up with them that frequently, at least with active study visits. Also, the need for masking. The more subjective the outcome, the more that either the patient or the evaluators can influence it, the greater the need for masking, and the type of masking, whether it's just the patient that's masked or the patient and the clinician that's masked. So all of this is influenced directly by the outcome that's chosen. Of course the personnel and resources required for the trial. You may need to have more sophisticated resources and more highly trained personnel if you're going to use some imaging procedures as an outcome. It certainly will influence the quality assurance procedures that are put in place to ensure the accuracy of outcome assessment. Whether that's going to involve site visits or can be done from afar, maybe by a panel of adjudicators or you know, reviewing samples from data collected during the study. It influences the cost of the trial and it may also influence the generalizability of results. How quickly could these results, or how meaningful are they, for clinical practice? So kind of in summary of those considerations is the three B's. How does the outcome related to the biology of the disease or condition that you're trying to change with the interventions? And does a change in the outcome reflect a really clinically relevant factor change? And the closer, usually, you get to clinical relevancy, the bigger the trial needs to be. But certainly, biostatistics in the power calculations. What's the detectable difference between the groups, that is, both plausible and practical? So what can actually, hypothosize to happen, without being wildly optimistic and is that a practical thing to be able to do? And the most important aspect of practical is usually the budget. Can you afford the total N for the trial, the total sample size that a particular outcome will require? And can you afford to measure it? Because reliably in every participant, do you have that equipment at all the centers, and can it be done in everyone? Or will participants refuse to have the measurement? So the choice of an outcome is usually a balance of these three. And I just wanted to go through another example that I hope will kind of clarify some of these points. So, if the trial is for evaluation of an antiretroviral treatment there are several measures that might be used. You could measure survival. Did the person, are they dead or alive? Did they progress to AIDS or not? You can look at the immunologic response, in the person. You could see what their CD4 counts were or other measures of how they're responding to H, the HIV disease, the virology response. It's very common to measure HIV viral load levels in trials. Or you could, you could be looking at the change in a patient's status. So, hows a patient? What's the quality of life? Because you may have a great immunologic and virologic response, but if they can't get out of bed, then that is not necessarily a good treatment. You may look at specific toxicities if there's a concern about that. And certainly in a lot of the more recent antiretroviral therapies there have been concerns about the effects of long-term toxicities that relate to cardiovascular disease, like elevated triglycerides. Or you may be just looking for other side effects, that could be one of the outcomes. So you can see for a particular treatment there could be a range of outcomes. And the choice of the primary outcome will depend on the objectives or the stage of the research. So, when you're in a phase one situation where the emphasis is on safety the outcomes that will be most important have to do with toxicity, specified or, or not. In a phase two study, where you're looking at short-term efficacy outcomes, you may be looking more for laboratory kind of surrogate outcomes like immunologic or virologic response to the treatment. If you were designing a pivotal trial that you were going to use to bring this to market, normally what we think of as a phase three trial, you'd probably be more interested in long-term efficacy measures, such as survival and also the toxicities associated with the drug. And then in a phase four trial, which is usually a post-marketing trial, you may be comparing two drugs to see which one is better, or two regimens, and in that you're probably interested in a lot of different measures: the survival, how the patient feels on the treatment, what their quality of life status is and the toxicities. And you may indeed be also interested in the influence of the treatments on short term measures of efficacy such as viral load and CD4 counts. So, regardless of the choice for a primary outcome, you'll probably in many situations collect data on other outcomes even when they aren't specified as the primary one. So how do we design trials to protect the primary outcome, to make sure were getting an accurate, reliable unbiased assessment of outcomes? Well, one thing, we protect against bias by randomizing people to different treatment groups, so that there isn't a propensity for one type of person to be put in a particular treatment group that may be related to the outcome. For example, putting the sicker people with the new exciting treatment. We also define the primary outcome, specifically in the protocol, and try to avoid any ad hoc definitions about what the outcome is. That should make you suspicious, if someone's trying to change the definition of the outcome, or what's included in the outcomes after all of the data have been collected, and maybe started to be looked at. So you don't want to do any on the spot definitions or post hoc selections of the primary outcome. That should be defined well before the data are collected and certainly before they are analyzed. You want to have standard methods for the measurement, and you want a measure that at a standard interval. So you want to have not only methods for the measurement but also a standard follow-up schedule so that you can avoid differential follow-up in the design of the trial or by default. And so what I mean in the design of the trial, if you are doing a surgery versus medical treatment trial, you may need to see those surgery patients more often, right after surgery, than you would need to see the patients assigned to medical treatment. But, in that case, you could either design a trial with a fair amount of follow-up in the beginning for both groups or, you could very clearly define what is clinical care and what's study follow up and make sure that you keep them separate and not start doing study measurements at a follow up that is really just for clinical care. So another feature we use to protect outcomes is masking of patients, clinicians, and/or the people evaluating the outcomes to the treatment. So they don't know when they're evaluating outcomes what treatment the patient is assigned to. You would try to limit observer variation and that's within an observer. So there is adrift over time so we can have some quality assurance that the person evaluating things isn't changing how they evaluate things over time and also to make sure that different observers have the same criteria for evaluation. and it's also important that you analyse all the events from all the patients enrolled. And when you start seeing publications that talk about evaluable patients, then that should raise a red flag, because why did they exclude events that happened in patients? And that could mean that they're trying to manipulate or, either purposefully or not, what is the overall interpretation if there's a specific type of patient outcome that's not included? So, we're still protecting the outcome in the analysis by using the intention to treat analysis, which was covered in another lecture.
