"... An accurate false positive rate could not be estimated because only 10 million clinical trials were simulateddue to computation capacity. " Only 10 million clinical trials were simulated! Classic! I overlooked that quote and concentrated instead on the due to computation capacity quote at the end.

You read through hundreds of pages of text and often don't find a nugget like that. This is the way to go have someone else annotate the reports and find the good ones. However, I do encourage everyone to read the full reports for themselves as there is much more informative discussion included, though admittedly none of it is as humorous as the above quotation.

The FDA ran out of CPU on its supercomputer trying to find a clinical trial in which aducan and placebo had similar clinical efficacy... and then had to give up trying after 10 million attempts with no luck ... will need time to requisition a larger supercomputer? OKKK. "Essentially impossible"? Well that can always throw cold water on further discussion on aducan's efficacy.

With small n statistical analysis I would typically not feel comfortable signing off on a p-value much less than 0.5% because as I have seen with the aducan data set even single patients can move the numbers around quite a bit. The FDA has computer modeling that they have confidence brings the p-values down to less than 1 in 10 million? Impressive.

It would as mentioned previously still be very helpful if they could provide more detail on the computer modeling that they did to arrive at that conclusion. The tables in the report mention slow, typical, and fast progressors; they also mention race and baseline MMSE. It is still surprising to me that APOE genotype, gender etc. were not seen as relevant predictors -- from merely looking at the reported numbers APOE +/- did seem to be important. Was this also a covariate effect?

I want to stay away from strictly following the aducan news cycle, however, it seems worthy to note that today the FDA announced that they would restrict the on label indication to MCI and early AD patients as were treated in the aducan trials. This does seem a reasonable adjustment and it was widely noted as unusual that the original label did not reflect the treated patient population. From the posts on forum approval of aducan did not seem overly surprising to me, yet as noted elsewhere it was not so much the yea or nea vote that would be most interesting but the conditionalities and nuances which would accompany the approval. Surprisingly, there were no conditionalities on approval: even amyloid scans appear to be optional.

[Edit: On more careful reading of the statement, this is not exactly what the update says.

"Aduhelm is indicated for the treatment of Alzheimer's disease. Treatment with Aduhelm should be initiated in patients with mild cognitive impairment or mild dementia stage of disease the population in which treatment was initiated in clinical trials. There are no safety or effectiveness data on initiating treatment at earlier or later stages of the disease than were studied."

Ohhh, this is going to be a much tougher sell than I had at first appreciated. Keyword is initiated. The statement as it reads above implies that those with moderate (or later) illness are still eligible for treatment; it is just that treatment with Aduhelm must be initiated before the moderate (or later) stage is reached. I had wondered how this would be applied: patients with Mild AD would receive treatment and especially those who achieved clear treatment benefit would then (what?) be denied treatment merely because they progressed even with clear disease slowing? Once you give toys to children, it is really hard to take them away again with a pitched struggle. The irony is that one could realistically imagine that APOE 33s who initiated aducan treatment even with moderate illness might have less risk of side effects than 44s with MCI. Amyloid levels tend to naturally decrease or at least plateau as AD advances.]

Even still I realize how difficult this will be for those who will no longer qualify for on label use. Many of the patients on the aducan extension trials have now advanced well into moderate AD and apparently continue to derive benefit from their treatment. Recently published results from DIAN appear to suggest that early treatment of anti-amyloids upwards of ~~10 years before onset are also helpful. Will they actually need to go through 5 years or more for phase 3 and other trials to establish that a known neuro-toxin should be removed from the brain? One bright spot here is that the dona phase 3 is recruiting down to 20 MMSE; the leca phase 3 (Clarity {which the clinical trials gov site has yet to update to active not recruiting since it completed enrollment in March!} is enrolling down to MMSE 22; gantenerumab phase 3 also down to MMSE 20. Makes one wonder what will happen when the patients who started treatment above 24 on aducan fall below 24? If this is all about playing by the clinical evidence, then patients who fell below 24 (even into moderate AD would still be eligible as per the extension inclusion criteria on clinicaltrials.gov.

Last edited by J11 on Thu Jul 08, 2021 9:32 pm, edited 3 times in total.

The top figure is from page 26 of the recently posted FDA pharmacology review. While I am clearly at a disadvantage trying to argue the point with those with their own supercomputer which they can max out on clinical trial simulations, I don't think that I would present the results as given in the top figure.

The report does appear to note the problems with presenting the trials in one figure and the adjustments that are then made.
In fact, the report later does conduct study by study regression analysis. Yet, separating the trials right from the start based on the
seeming pattern of a low group (aducan, leca etc.), and a high responder group (103, APOE+ and Mild AD etc.) one can achieve substantial insight into the nature of the responses. In particular, the high responder group demonstrates very significant responses. On this regression line, patients experience CDR-sb benefits of ~1!

"Because of differences in inclusion/exclusion criteria for study 103 and 301/302, there may be differences in demographics and baseline characteristics, in addition to obvious differences in the sample sizes. Therefore, when evaluating SUVR – CDR-SB relationship based on data across different studies, it is necessary to use placebo-corrected, baseline adjusted values to account for the differences or imbalances between studies. When placebo-response is different between studies, not correcting for placebo-effect (i.e., using only baseline-adjusted values) can affect the interpretability of the underlying relationship between SUVR and CDR-SB. This approach of using placebo-corrected, baseline-adjusted values when exploring potential relationships between biomarkers and clinical endpoints across different studies is well known and has been employed in regulatory assessments previously."

Nevertheless, I still think that it would be helpful to add in the placebo point; still I am unclear on this point because it might already be understood because everything else is measured relative to it. Being explicit about where the origin is seems helpful to me.

I also think it would be best plot the points as reported. The reports speak of adjusting for baseline and placebo. This does seem reasonable, however, then 301 high is way way off the chart. Plotting the as reported values brings the points more into alignment as seen in the middle figure. The 302 high moves somewhat upwards; 301 moves much farther down ...

It would also be best to plot the 103 trial separately. 103 had large clinical benefit likely resulting from using a different patient mix. The 301 and 302 trial appear to have used a more typical memory clinic standardized MCI/Mild AD patient and the results for many of the other anti-amyloids then fall neatly along a single regression line. As seen in the bottom figure above the 103 trial have strong linear correlation (94%) with a slope of ~4 versus the more typical result of ~1.3 found for most of the other trials. 103 and the rest of the trials seem to have clearly different outcomes. One lingering confusion with 103 is there are two sets of results (e.g., MMRM) which produce 2 different regression slopes.

The report then goes on to note the importance of thinking in terms of these group correlation plots instead of individual level correlations. Individual level data were not randomly assigned to different SUVR levels in the same way that group level was carefully controlled and balanced. This is a very important idea that caused a great deal of confusion on thread and in the general conversation about aducan.

Last edited by J11 on Fri Jul 09, 2021 11:15 pm, edited 2 times in total.

These are more great figures that are also similar to many of the correlation plots posted to the thread. I will have to double check some of my plottings because these figures appear to place the points differently. For example, the gantenerumab clinical trial results are shown as being above the 0 CDR-sb line. In my plots they were below and highly aligned with the regression. The points in the above figures are for the 104 week point; I will check what week and what particular study my points referred to. The top figure also included all 5 arms of the Ban2401 phase 2. I had thought that there were only 2 treatment arms with SUVR and CDR-sb info. Yet, recently, I saw the publication had all 5 arms given. What I am unsure about is that these arms had smallish number of patients and some were just not plausibly correct; they did not have good covariate balancing etc.. Admittedly this is somewhat distorting to only take some of the data that fits the pattern that you see, even still I do not see any great value in loading up a figure with datapoints that are clearly wrong. With 301, the point was in rapid evolution even from the interim analysis to the final analysis 3 months later. Presenting a static point and implying that this is somehow truly representative of where it should be placed also seems highly deceptive.

What is also of interest is that in the bottom figure above, the dona phase 2 is much closer to the 302 high dose result than in my figures. In my figures, dona was placed at ~ -0.37 SUVR versus ~ -0.28 for aducan. This seems like a large distance. However, in the figure above, instead of SUVR change the x-axis is difference from placebo on percentage change in centiloid scale (%). Here aducan achieved -77% while dona achieved ~ -87%. It is not obvious to me why the scaling on the x-axis would need to be changed in this way. All the other anti-amyloids were placed on a simple SUVR difference scale.

{Great! The mystery about dona might now be solved. The dona phase 2 result had been the only modern era anti-amyloid (aducan, leca, dona, ganta) which was somewhat off the regression line. I had been quite unsure why this might be. However, on pages 28-29 of the pharma review there is a discussion of how the phase 2 with dona had patients with more amyloid. dona had 108 centiliods versus 80-90 in the aducan p3s. So, dona was struggling against a stronger head wind than aducan. This would have the effect of moving up (dereasing the CDR-sb benefit) the dona point away from the regression line. To counter this disadvantage the point should be corrected by moving it downwards or alternatively it could simply be moved rightward as less relative amyloid is removed.

I had been very unclear why dona seemed to be not behaving as all the other anti-amyloids were behaving. The reasoning here might be that in order to select on tau patients with more advanced AD on average would be needed-- these would not be the typical memory clinic trial population. As was noted previously, in making this tau selection it was also found that the placebo decline decreased while the percentage reduction in decline increased to ~50% in the lower tercile of the intermediates. All the other trials simply enrolled the ordinary MCI/Mild AD patients that existed. This will be a tricky one to place correctly, though at the very least a highly reasonable explanation for the discrepancy has been found and the dona point should now be in good alignment with all the other points.}

I had not realized that sola reported such strongish results. If there were an additive (or better?) combination with aducan, then this could be another win for patients. A great many anti-AD drugs from the past might be reanimated in this new world where amyloid is brought under control. Once amyloid is managed, treatments that previously had to fight an uphill battle might have a much easier time showing their positive effects.

Yes, more news today -- too much news!
Less news more data analysis.

The above figures?
I noted recently that the lecanemab phase 2 had more datapoints that had not been accounted for on the thread's correlation plots-- some of these points seemed to be "off" of the expected regression line. I wanted to understand how/why the points were not fitting the line. These points are shown as gold triangles a few posts up.

The first figure in the series above shows the straight as reported result for the phase 2. As we can see the three points to the left of the placebo do not fit the line well, while the 2 points on the left that were included in this thread's regression plots are very much as to be expected by other anti-amyloid results. The 5 mg/kg dose arm and the 2.5 mg/kg bimonthly arm are especially "off". The slope here is 1.1 and R^2 is 0.288 which are somewhat disappointing, given the other results.

However, the next in the series gives somewhat better results. This time each point is weighted by its size (~ modulo 30). Size weighting is the preferred method. Slope does not change much but now R^ =0.56.

Up till now we have used the MMRM frequentist numbers as presented in the published report. The trial though was driven by a Bayesian analytic framework. What did the Bayesian results for CDr-sb find? The next figure shows the results for an unweighted Bayesian regression. The slope has now increased to 1.25 and R^2 is 0.62.

The next figure uses the preferred method of size weighting with the Bayesian results. We are now at 1.30 and R^2 = 0.80. This keeps getting better and better!

Now what we can do is "reduce" the data points that have the same monthly dosing (also weighted by sample size). For example, the 2 points to the left of placebo have treatment dose of 2.5 mg/kg *2 times per month or 5 mg/kg *1 time per month. What is of interest is that these two doses actually are the same monthly dose. In fact, they remove almost the same amount of amyloid (with the 5 mg/kg once per month removing slightly more). Combining the two doses by weighting the SUVR and CDr-sb values should give us a better estimate because it will increase the smallish sample sizes of around 35 each. Same reduction can be done with the 10 mg/kg once per month and the 5 mg/kg twice per month. Nearly same amount of amyloid is removed and the sample size can be increased. When this is done as can be seen in the middle figure of the above post the R^2 increases to 0.87.

Finally in the bottom figure in the above post, we can "reduce" the data points, weight them by treatment arm size, and use the Bayesian values. Startlingly, we now have a slope of 1.28 and R^2 = 0.978. Funnily enough this is r= 0.9888 and when I plugged this into an online correlation calculator it finds that for 3 points this has a p-value of 0.095 and for 4 points a p-value of 0.012. I am unsure whether the placebo really should be recognized as an independent point. It surprises me that with such a large correlation (~0.99) the p-value is not somewhat higher. How many times could one throw a dart and have such strong collinearity?

Probably best not to overanalyze this one because this was another in the series of very confusing AD trials. The Bayesian model was scuttled part of the way through as APOE+s were no longer allowed into the highest most effective dose group (10 mg/kg *2). This dosing arm then became overweighted with APOE-s; the APOE+ were then overweight into the 10 mg/kg monthly and the 5 mg/kg biweekly arms. The covariates while supposedly corrected for in the MMRM model had large imbalances with respect to MCI/Mild AD, APOE ... Nevertheless, the final figure above with a weighted Bayesian reduced linear regression has a result extremely in line with previous findings.