![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
temporaryreality) wrote2021-08-11 09:13 pm
IM Doc pt4
Further Discussion of the Red Flags in the Pfizer Vaccine Paper in the New England Journal of Medicine
Earlier this week, we posted An Internal Medicine Doctor and His Peers Read the Pfizer Vaccine Study and See Red Flags [Updated]. Most readers responded very positively to the write-up by IM Doc, which included the reactions of the eight other members of his Journal Club who reviewed the article and its editorial, as they have done regularly with important medical journal articles. We have embedded the Pfizer article from the New England Journal of Medicine (NEJM) below; the link to the editorial is here.
However, some took issue with IM Doc noting that two nurses in the UK had suffered anaphylaxis, a severe, potentially life threatening allergic reaction, after getting the Pfizer shot. IM Doc criticized the paper and editorial for not including or adding a discussion of any exclusion criteria, particularly since Pfizer’s proxies admitted that severe allergies were an exclusion criterion. From MedicalXpress:
Moncef Slaoui, who is the chief advisor to the US program for COVID vaccine and treatment development, told reporters, “Looking into the data, patients or subjects with severe allergic reaction history have been excluded from the clinical trial.
“I assume—because the FDA will make those decisions—that tomorrow this will be part of the consideration, and as in the UK, the expectation would be that subjects with known severe reactions, (will be asked) to not take the vaccine, until we understand exactly what happened here.”
Slaoui is the co-head of Operation Warp Speed and previously head of GlaxoSmithKline’s vaccine department. Other media outlets and professional medical writers (see here and here for examples) picked up his statement that subjects with severe allergic reactions were excluded.
If you look at the article below, you will see that it is not searchable. That indicates an expectation that it would be read as a print out only. You will find it make no mention of “exclusion criteria”. Neither does the the separate editorial by NEJM editors. The article does does mention “protocols” in the text, twice, but does not have a link to where to find them, does not have a written URL, nor does it provide a name or location to assist in finding them.
Some critics argued that the protocol (which you need to search through to find the selection process for candidates, including the exclusion criteria, for the Phase III trials) could “easily” be found in the Supplemental Materials and further asserted that any regular reader of medical papers would be able to find then. The fact that IM Doc, who has been reading medical papers for 30 years, and his eight colleagues did not locate them is already significant counter-evidence, particularly since the NEJM’s media kit lists the publication’s audience solely as physicians. No doubt scientists read it too, but the eyeballs advertisers really want to reach are doctors, academics or scientists in the employ of competitors.
IM Doc could not find the Supplemental Materials because the PDF that the NEJM generates does not include them. It is in the online version, and opens up to a dropdown menu, with the first item “Protocols” which takes you to a document via an external link. Since readers have every right to assume that online and PDF versions of the same article are identical, there was no reason for him to look further.
It turns out that the data waters appear to have been muddied by the NEJM itself. IM Doc and his colleagues found and read the Pfizer paper late last week. It was then on the first page of the site. He sent me his write-up on Saturday the 12th. I went to look at the article and charts on the NEJM site. It was then on the front page of the site. I experienced rendering issues in Firefox but nevertheless was able to look at the article, along with the separate tab in the header area for “Figures/Media”. I also noticed a “Supplemental Appendix,” which I opened. It was a bit of a hodge podge but didn’t contain anything that related to IM Doc’s observations. I did not see any tab with “Supplemental Materials.”
IM Doc did a big revision of his draft on Sunday, which I edited that evening. IM Doc was a bit freaked out Monday AM when the Pfizer article has moved off the NEJM front page, but he quickly located it on the site. I looked quickly, simply registering that I could find it and see the charts, but I did see that the rendering problems were no longer occurring.
I was not the only person who recalls seeing the “Supplemental Appendix” (notice NOT “Supplemental Materials”) as a stand-alone document with a link to it in the top area of the NEJM site before December 12 (sadly, the Wayback Machine allows publishers to suppress older versions of articles upon request, so there is no image of the article as it initially appeared on the first page of the NEJM site).
Fortunately, reader KLG harrumphed about the Supplemental Appendix in comments. KLG is a professor of microbiology and has been doing basic research for 30 years, so he can’t be dismissed as inexperienced in reading scientific papers. I asked for his recollection of what he found when he went to the Pfizer article:
Here is how I remember finding the paper, after seeing the post from IM Doc yesterday. I apologize if this is TMI, but I want to be as clear as possible:
(1) I clicked on the link while in my office and gained direct access through our medical library, downloaded the pdf, and printed it on a high-resolution color printer, which is my standard, old-fashioned practice. I then read through the paper fairly quickly, and thought it was OK/promising but not necessarily complete.
(2) I saw no obvious (printed) link in the pdf to any supplementary materials, so I went back to the online link to the paper through our library (full access to most medical journals). I saw the link to the 12-page “Supplementary Appendix” with the 4 pages of names and downloaded and printed it. This link was on the right side of the screen/page, near the top. This link was not at the bottom of the single page of the paper as I see NEJM.org from home this evening, along with the other links, including Protocols, Disclosure Forms, etc. I am not accessing the journal remotely through our library tonight and would be seeing it as “outsiders” do. The point for me is this: One link to supplementary material should go directly to all supplementary materials. This has been what I have been accustomed to for years. Sometimes the files are ridiculously large and there may be 10-15 of them, but they are all there and easy to find. If one link cannot be managed, then all links should be in the same place on the webpage, listed one after another as S1, S2, S3, etc.
(3) Still, I may well have missed these other links, because while I am very skeptical of “science” direct from Big Pharma, my forensic antenna were not fully deployed until the usual suspects showed up later in the day as I checked back in to see how things were going.
(4) But more importantly, I do not believe for a second that IM Doc and his like-minded journal club (a common mechanism for all biomedical scientists and many clinicians to keep up with current developments) would have missed these materials, if the links were properly displayed as they should have been on day-one. As I mentioned in my previous email, so-called supplementary material has become a thing, for good or ill, in biomedical publishing, and in my experience the links are prominent in strong journals (and NEJM is definitely that, or certainly was when Marcia Angell was editor). Moreover, IM Doc undoubtedly has a subscription to NEJM, which should have displayed the links prominently both in hard copy and online. Based on his every contribution to NC, IM Doc seems uncommonly attuned to both the practice and science of medicine, going back to the beginnings of the HIV/AIDS epidemic, which made a deep impression on all of us who were there, clinicians and non-clinicians alike. That is when we learned to parse the literature and separate the wheat from the chaff. And there was a lot of chaff in the HIV/AIDS literature from ~1983 until triple-therapy was published in 1994 IIRC and AIDS became a chronic, manageable condition for most of those infected with HIV.
Let us stress again, as you can see from the PDF below, which I downloaded Monday morning: In the PDF, there are no Supplemental Materials nor any reference to them. The Section is completely omitted. This is where the link to the Protocol would be found, were NEJM to have followed its usual practice in other articles.
In other words, NEJM initially either did not present the Supplemental Materials tab at all or through terrible design, directed reader attention away from it by having a prominent Supplemental Appendix link at the top, which experienced readers would assume contained all of any additional documents, save any others mentioned and/or linked to in the article proper and/or the editorial. And even when it cleaned that up, NEJM did not update the article to include a link or printed URL, or a reference.
In addition, even after readers in comments had pointed out where to find the exclusion criteria (in the Supplementary Materials, in Protocols, meaning two clicks and a search, when it should have been easier to find), other had difficulty finding it and also deemed the material to be poorly presented:
Thank you for highlighting the exclusion criteria. When I looked at the NEJM article, I was also unable to find them. I noticed that a reader has since posted them, and you updated the post. I also found the on Pfizer’s website. https://www.pfizer.com/science/find-a-trial/nct04368728-0
My wife is a graduate of Stanford Medical School and is a subspecialist in Internal Medicine. Not infectious disease. She was at the top of her class. She graduated a bit more than 10 years ago. I was so intrigued by this doctor’s post that I wanted to do a little test, admittedly with just one subject. As you can see I was an early commenter and have skin in the game because I live with a doctor.
I wanted to see if she could identify quickly the exclusion criteria just like this doc has now admitted was confusing but actually there. She could not do so quickly – and it took her about 45 minutes to do so and then she stated to me when looking through them how recklesly they were spelled out. She was surprised. Phase 1 stuff all mixed in with the others. She was also very concerned about the fact that there were so many common conditions that had not been studied or included in a wide population impact drug like a vaccine. Concerned about why these are not being more publicized given the fact that we will be trying to vaccinate presumably everyone. Should this not be publicized more with both docs and patients.
This discussion should put the question of the adequacy (not) of the NEJM presentation of information about the Pfizer vaccine to rest. It was incumbent on the New England Journal of Medicine to provide a concise overview and ready access to details important to practitioners to help them assess a medication intended for the entire population, yet developed and approved on a corner-cutting basis. They fell short.
Lambert, who in a prior life was a document analyst and developed schema for medical journals, was taken aback by other data presentation failings in these pieces. He intends to discuss them in the near future.
[to see what was inserted here, please go to the original post, linked via the orange title]
IM Doc
A few words about your link about “Antibody-Dependent Enhancement”
My thinking process about COVID vaccines is definitely affected by my access to all kinds of conferences and emailed insights from previous academic colleagues all over this country. Not unlike AIDS during the 1980s, there are fathoms that we simply do not know about this virus. And unfortunately, unlike AIDS, we now have the internet and social media that can spread things not proven or true like wildfire.
This concept of antibody-dependent enhancement (ADE) is one such issue. I am seeing this twisted into Armageddon all over the place online – and I wanted to give at least some persepective.
I think this article you linked is as accurate as we can be right now based on what I have been reading and listening to from people who know. And it has the benefit of not being written in medical jargon. I encourage everyone to read it.
Since the beginning of this Operation Warp Speed, I have heard the same three large concerns from specialists from vaccinology, virology, immunology, and infectious disease. There have been many other concerns about the vaccines and our approach – but these three distill all the important stuff.
— The rapid fire testing is likely going to lead to severe safety issues some expected and some out in left field. We have got to get the doctors of America ready for this. ( A CLEAR FAIL – so far).
— This applies only to the mRNA vaccines- they have failed in all previous human trials, PHASE I Zike and Influenza, because of the predilection to produce anaphylaxis and other allergies in a statistically significant number of subjects. If this begins to happen immediately upon deployment, we know that we may have a problem. This is important – Because of the nature of this emergency, the risk/benefit ratio may very well be skewed toward continuing forward – only time will tell. And this should be a society decision based on transparent and correct data points.
— Thirdly, ADE is indeed a very real concern, and has scuttled coronavirus vaccines in the past in animals, and most certainly was responsible for the DENGIVAX disaster a few years ago. This problem will require already vaccinated patients to come into contact with the wild virus. It will take likely months for us to have enough of a cohort walking around to know if this is a problem. I want everyone to understand, there are immunologists and virologists everywhere working tirelessly to see if evidence of this is emerging in tissue samples and immunologic evals already. Again, only time will tell. But this is not going to be immediately obvious like the anaphylaxis issues already are. What is of concern to many informed people, is the fact that we are seemingly immunizing our entire health care system. If this ADE problem does come to the fore, we could be in big trouble. Those who are recommending holding back a portion of the health care workers are being opposed by those who are concerned this holding back will alarm the public and drop already weakened vaccine confndence in the general public. (These issues are profoundly difficult to understand and to explain for everyone). This is a Gordian Knot – and I am not certain anyone has the answer.
Bottom line – that ADE article is very good – it is comprehensive – and will keep people informed at least where we are now. When I was in medical school, the immune system was treated as a turnip. It is becoming increasingly clear that it may be the most complicated biological system in the universe. It makes your eye, for example, look like a 5 year old’s Lego set.
Have a Wonderful Holiday Week everyone.
> it has the benefit of not being written in medical jargon. I encourage everyone to read it.
Derek Lowe is very good, and he writes in English. Unfortunately, he doesn’t have a newsletter, so I have to remember to go check him.
one thing. ADE is not the only concerning immune reaction that we need to pay attention to. In fact others may be more problematic.
ADE in respiratory infections is included in a broader category named enhanced respiratory disease (ERD), which also includes non-antibody-based mechanisms such as cytokine cascades and cell-mediated immunopathology