COVID OUT CRITIQUE

Note this critique was compiled by the c19early.com research group at ivmmeta.com and does not represent my original work.

COVID-OUT.

Randomized Trial of Metformin, Ivermectin, and Fluvoxamine for Covid-19

COVID-OUT remote RCT, showing no significant differences compared to a combined metformin/placebo "control" group. Results for other treatments are listed separately - metformin, fluvoxamine.

Authors include metformin patients in the control group, allowing details of adjustments to affect results. Using standard treatment vs. placebo analysis shows 61% lower hospitalization, or 75% lower for patients with onset ≤5 days (not statistically significant with only 7 and 5 events). These results are not reported in the paper or the supplementary appendix, readers need to request the data. Authors note that "hospitalization is perhaps the most accurate and well-documented end point".

There are many major issues as detailed below. We provide more detailed analysis of this study due to widespread incorrect press. Submit Updates or Corrections

SeverityIssueCRITICAL1. Ivermectin vs. placebo analysis - 61% lower hospitalizationCRITICAL2. Severity mismatch for ivermectin treatment but not for any other medication or controlCRITICAL3. ER results unreliable, not related to symptomsCRITICAL4. Mismatch with reported death and symptomsCRITICAL5. Ivermectin vs. placebo symptoms consistent with efficacyCRITICAL6. Multiple outcomes missing, including time to recoveryCRITICAL7. Hypoxemia results unreliable but prioritizedCRITICAL8. Adverse events not reported, partial data shows no increaseCRITICAL9. Control group includes metformin, adjustment protocol violationCRITICAL10. Viral load results missingCRITICAL11. Metformin/fluvoxamine conclusions opposite of Together Trial, but matching earlier studies on each teamCRITICAL12. Author claims results from 596 researchers should be censored for false informationCRITICAL13. Administration on an empty stomachCRITICAL14. Results delayed 6 months (including life-saving metformin results)SERIOUS15. Fewer comorbidities for serious outcomesSERIOUS16. Control arm results very different between treatmentsSERIOUS17. COVID-19 specific symptoms hidden in appendixSERIOUS18. Authors claim placebo is not better than the treatmentsSERIOUS19. Incorrect claim that no treatment reduced severitySERIOUS20. False conclusionSERIOUS21. Trial outcomes modifiedSERIOUS22. Very high percentage of missing dataSERIOUS23. Medication delivery varied significantlySERIOUS24. Treatment 3 days for ivermectin, 14 days for metformin and fluvoxamineSERIOUS25. SAP dated after trialSERIOUS26. Test requirement and delivery prohibits early treatmentSERIOUS27. Conclusion modified by journalSERIOUS28. Symptom results contradictorySERIOUS29. Adherence very lowSERIOUS30. Inconsistent blinding statementsSERIOUS31. Author indicates a best guess can be used for onsetMAJOR32. Ivermectin from source chosen has shown lower efficacyMAJOR33. Adherence subgroups analysed but not reportedUNKNOWN34. Maximum symptom duration not clearUNKNOWN35. No discontinuation due to hospitalization for ivermectinCOMMENT36. Authors indicate up to 5 day delay in real-world usage

Author responses14. Results sent to the US government twitter.com (BC). Note most people live outside the US, and there was no action.No response for all other items

Ivermectin vs. placebo analysis - 61% lower hospitalization. Authors include metformin patients in the control group, allowing details of adjustments to affect results. Using standard treatment (ivermectin only) vs. placebo analysis shows more favorable results for ivermectin, with 61% lower hospitalization, or 75% lower for patients with onset ≤5 days (not statistically significant with only 7 and 5 events). Authors note that "hospitalization is perhaps the most accurate and well-documented end point".

Severity mismatch for ivermectin treatment but not for any other medication or control. The table shows the percentage of patients reporting severe dyspnea for each active treatment and respective control. We expect that patients reporting ER visits would be more likely to experience severe dyspnea. This is true for all cases except for ivermectin treatment, suggesting unlucky randomization for ivermectin treatment, or a potential data error. The percentages are with respect to the total number of patients reporting symptom data in each case.

Ivermectin activeIvermectin controlMetformin activeMetformin controlFluvoxamine activeFluvoxamine controlER0.0%9.1%13.3%10.0%10.0%14.3%Non-ER6.1%6.9%8.0%7.8%6.4%8.4%

ER results unreliable, not related to symptoms. Authors detail why the main hypoxemia results are unreliable, however the ER results appear to be similarly uninformative. ER visits do not appear to be related to symptoms. The mean total COVID-19 symptom score for patients reporting an ER visit is 55 compared to 56 for patients reporting no ER visit (or hospitalization/death). Visualization of the ER patient symptoms raises the question of why most of them went to the ER. Of the 26 patients reporting an ER visit and symptom data, only one ever reported severe dyspnea, 5 more reported at most moderate dyspnea, 11 more reported at most mild dyspnea, and 9 reported no dyspnea at any time. ER patients were less likely to report severe or moderate dyspnea. The decision to go to the ER appears to be more of a personal preference rather than based on symptoms. Patients that signed up for the trial may be especially concerned about PASC for example, and seek help based on potential future problems rather than current symptoms.

Maximum dyspnea severityER patientsNon-ER/hosp./death patientsSevere3.8%6.5%Moderate19.2%22.2%

Mismatch with reported death and symptoms. There was only one death for a patient that was treated very late (7 days). The patient was not hospitalized. The death is reported within 14 days, however the patient reported symptom data for all 14 days, showing substantial recovery several days prior, with only 2 of 14 symptoms remaining and reported as mild. Data suggests that the death was not due to COVID-19.

Ivermectin vs. placebo symptoms consistent with efficacy. Authors include metformin patients in the control group, allowing details of adjustments to affect results. Using standard treatment vs. placebo analysis gives the mean COVID-19 symptom scores below, matching expectation for an effective treatment with the low-risk fast recovering population (note that administration on an empty stomach is expected to delay the time when therapeutic effects may be reached).

Multiple outcomes missing, including time to recovery. Multiple outcomes are missing, for example time to recovery (where ACTIV-6 showed superiority of ivermectin): "Time to meaningful recovery (symptoms or severity improved by one category and sustained for at least 36 hours)" (protocol page 91). Notably, the definition is less biased than the ACTIV-6 definition, including improvement by one category, making allowance for mild fatigue and cough, and requiring 36 hours sustained rather than 3 days. More notably, the result is not reported.

Hypoxemia results unreliable but prioritized. Authors detail why the hypoxemia results are unreliable vimeo.com (E) @28:30, however they are still prioritized in the presentation, and included in the abstract without mentioning that these results are unreliable.

Adverse events not reported, partial data shows no increase. Adverse events are not reported (other than to note none were serious). Partial information is contained in Table S2 and Figure S5. Notably, there is no significant increase for ivermectin for any of the expected side effects, in contrast to other trials, e.g. Lim. These results are unexpected if patients received and took authentic ivermectin at the dosage indicated.

Control group includes metformin, adjustment protocol violation. The "control" group includes patients receiving metformin, which is known to be beneficial for COVID-19 c19early.org (C). Authors present adjusted results however they do not appear to fully account for metformin efficacy. For example, the adjusted result for ivermectin ER/hosp./death is close to the unadjusted result, while a greater difference would be expected based on the metformin efficacy reported (which is not expected to be doubled in the metformin + ivermectin arm). The trial has 5 treatments arms, but is presented as if there was 3, which adds complexity, makes the results subject to potential interactions between treatments, and introduces the potential for investigator bias in adjustments. Notably, the protocol specifies primary and secondary adjustments (page 74), and the paper reports only one set of adjustments, which matches neither the primary or secondary adjustments in the protocol.

Viral load results missing. One of the primary objectives of the trial was "To understand if the active treatment arms are superior to placebo in improving viral load, serologic markers associated with Covid-19, and gut microbiome in non-hospitalized adults with SARS-CoV-2 infection." In the July 8 presentation authors report having analyzed viral load data vimeo.com (E) @19:20. No viral load results have been reported.

Metformin/fluvoxamine conclusions opposite of Together Trial, but matching earlier studies on each team. The Together trial and COVID-OUT both tested metformin and fluvoxamine. Notably, they came to opposite conclusions. In Together, authors found efficacy for fluvoxamine, but the metformin results were so negative that the trial was terminated early. In COVID-OUT it was the opposite, authors (although not the journal editor) found efficacy for metformin, while the fluvoxamine results were so negative that the trial was terminated early twitter.com (Y). Note that the Together authors include researchers that found fluvoxamine effective in earlier studies, while the COVID-OUT authors include researchers that found metformin effective in earlier studies.

Author claims results from 596 researchers should be censored for false information. 60 studies by 596 scientists report statistically significant positive results for ivermectin treatment of COVID-19 c19ivm.org (F). One author claimed that a report of positive results is "disinformation" and distributed a request to report and censor the author twitter.com (AA), twitter.com (AB), twitter.com (Z). While discussion is warranted for all studies, a call for censorship of results is extreme and raises questions. Author provides no basis for the results of the 596 scientists being wrong and warranting of censorship, and there is no indication that author has even read most of the studies. Author cherry-picked two of 99 studies, (COVID-OUT and ACTIV-6 Bramante, Naggie, both very high COI studies with an extensive list of issues and very delayed treatment) and claimed that "no benefit of ivermectin was observed" twitter.com (AC). In addition to ignoring the 60 studies reporting statistically significant positive results, ACTIV-6 c19ivm.org (G) reported a posterior probability that ivermectin is effective of 99%, 98%, and 97% for mean time unwell, clinical progression @14 days, and clinical progression @7 days (even though none of the pre-specified primary outcomes were reported, and noting that these preprint results were changed without explanation), and COVID-OUT showed 61% lower hospitalization with ivermectin vs. placebo (not including metformin), although this was not reported.

Administration on an empty stomach. Authors instructed patients to take ivermectin on an empty stomach, but other treatments with food. Guzzo show that the plasma concentration of ivermectin is much higher when administered with food (geometric mean AUC 2.6 times higher). "Ivermectin or matching placebo should be taken by mouth on an empty stomach with water. 1 hour before or 2 hours after a meal. All other agents should be taken by mouth at the end of a balanced snack or small meal."

Results delayed 6 months (including life-saving metformin results). Results were delayed for 6 months with no explanation, with followup ending Feb 14, 2022. Results were not presented until July 8 rethinkingclinicaltrials.org (B), and they were still not available to the public due to a news embargo for over a month. Embargo and delay of clinical trial results during a pandemic is not consistent with a goal of minimizing mortality and morbidity. Notably authors report very positive results for metformin (although journal editors changed the conclusion as below).

Fewer comorbidities for serious outcomes. Patients experiencing serious outcomes are expected to be more likely to have comorbidities, however the opposite is seen.

OutcomeComorbidity prevalenceNon-ER/hosp./death53%ER45%Hospitalization12.5%Death0%

Control arm results very different between treatments. Control arm results are very different between treatments, for example considering hospitalization/death, this was 1.0% for ivermectin treatment vs. 2.7% for metformin control, however it was 1.3% for the ivermectin control. The metformin arm started earlier, however the difference in outcomes is very large given that most patients are in the shared period.

COVID-19 specific symptoms hidden in appendix. Authors present results for all symptoms in Figure 2, and for COVID-19 symptoms in the appendix Figure S4. Notably, the COVID-19 specific results are better for ivermectin and especially for fluvoxamine.

Authors claim placebo is not better than the treatments. Authors state: Neither overall symptoms nor Covid-19–specific symptoms were reduced faster with placebo than with any of the trial drugs. This may be true, Figure S4 shows symptoms were reduced faster with all treatments (with ivermectin and fluvoxamine showing greater improvement than metformin), but the reverse claim is very unusual — placebo is not expected to be better. Note that the graphs and data refer to the control groups including other treatments, while the statement refers to placebo only.

Incorrect claim that no treatment reduced severity. Authors claim that "None of the trial drugs resulted in a lower severity of symptoms than identically matched placebo." The intended meaning — compared to the "control" groups used, since that is the data reported — is incorrect, multiple results show lower severity in the treatment groups in terms of the symptom scores and severity resulting in hospitalization. Individual results may not reach statistical significance, however ER/hosp./death does in the larger metformin group.

False conclusion. Authors claim "None of the three medications that were evaluated prevented the occurrence of hypoxemia, an emergency department visit, hospitalization, or death associated with Covid-19." Taking the literal wording, this is false, there were no deaths with fluvoxamine. Taking the likely meaning (no treatment reduced incidence of these events), this is false, reduced incidence is seen in several results (mostly without statistical significance).

Trial outcomes modified. Trial outcomes were changed on January 20, 2022 clinicaltrials.gov (H), and again on March 2, 2022 clinicaltrials.gov (I).

Very high percentage of missing data. There is a very high percentage of missing data. 25% of patients have zero symptom data reported for all 14 days in the data file. This does not match the paper which reports 20% of patients did not contribute symptom data (Figure 2).

Medication delivery varied significantly. Medication delivery varied significantly over the trial. In this presentation vimeo.com (F), author indicates that delivery was initially local, later via FedEx, was much slower in August, there were delays due to team bandwidth issues, and they only realized they could use FedEx same day delivery in September.

Treatment 3 days for ivermectin, 14 days for metformin and fluvoxamine. Treatment was 14 days for metformin and fluvoxamine, but only 3 days for ivermectin.

SAP dated after trial. The SAP is dated February 14, 2022, which authors note is one day before unblinding. However, the protocol notes that the statisticians are unblinded: "There is one unblinded statistician with two unblinded supporting statisticians on the study team", and "All analyses will be carried out by the un-blinded statisticians". The protocol also notes that the SAP will be developed by unblinded statisticians in one case, and blinded in a second case: "detailed statistical analysis plan will be developed by the unblinded statisticians", and "statistical analysis plan will be developed by the blinded statistician."

Test requirement and delivery prohibits early treatment. The requirement for a positive test and delivery of medication introduces substantial delay and largely excludes the possibility of early treatment. The protocol requires verifiable results using a local laboratory standard which excludes most home antigen tests (supplementary data page 5). Note that the trial results do not generalize to real-world usage, where clinicians recommend treatment immediately on symptoms.

Conclusion modified by journal. Author statements indicate that the conclusion was modified by the journal twitter.com (BD), twitter.com (BE).

Symptom results contradictory. Authors consider only metformin results to be positive (the journal editor considers none to be positive), however the symptom results in Figure S4 show the opposite: ivermectin and fluvoxamine show faster improvement (without statistical significance), while no difference is seen for metformin.

Adherence very low. Adherence was very low, with 77% overall reporting 70+% adherence, and 85% for ivermectin reporting 70+% adherence. An author has claimed 85% took all doses but that is contradicted by the 20% reported "Total Interruption or Discontinuation" in Table S2. Numbers for 100% adherence are not provided.

Inconsistent blinding statements. Protocol page 12 states that "The research team statisticians will remain blinded", while the supplementary data page 40 states that "There is one unblinded statistician with two unblinded supporting statisticians on the study team".

Author indicates a best guess can be used for onset. One author suggests that investigators can use a "best guess" if a patient gives a range for time of onset, which would allow a biased investigator to present an incorrect lower average time from onset twitter.com (BF).

Ivermectin from source chosen has shown lower efficacy. Authors chose to source ivermectin from Edenbridge, which ranked 7 out of 11 brands in In Vitro tests for antiparasitic efficacy Williams, requiring 5 days compared to 2 days for the best performing brand, and 3 days for 4 other brands.

Adherence subgroups analysed but not reported. Authors indicate they performed subgroup analysis by adherence vimeo.com (E) @18:30, however these results have not been reported.

Maximum symptom duration not clear. The procol excludes patients with >7 days of symptoms, i.e. patients 7 days from onset are included. The paper claims "less than 7 days" in one instance and "within 7 days" in another. The presentation reports "<7 days" vimeo.com (E).

No discontinuation due to hospitalization for ivermectin. Table S2 shows 9 placebo patients discontinued treatment due to hospitalization, compared to zero for ivermectin. While ivermectin patients only received 3 days treatment, they received placebo tablets for the remaining days. If this number is only counting discontinuation during the first three days, the result highlights that treatment was stopped before any patients were hospitalized. The protocol notes "Study drug will be stopped at the time of hospitalization for any reason".

Authors indicate up to 5 day delay in real-world usage. Authors note up to 11 days treatment delay with a remote clinical trial compared to up to 5 day for "real-world use" vimeo.com (E) @43:00, where the 5 days derives from testing and medical system delays. However, logical real-world use, as used in many locations, is to have the treatment on hand to take immediately.