Draft synthesis: Tirzepatide compared with semaglutide in… — submissions
The 7 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
7 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
The same concept is given three different names in one document
This submission addresses the draft synthesis on In adults with type 2 diabetes, does tirzepatide achieve greater glycaemic and weight effect than semaglutide? from the standpoint of a reader who will use the summary and not the appendices.
The draft refers to the same quantity as a response rate, a responder proportion and a categorical outcome in different sections. The respondent, who works in health-technology assessment, states that a reader cannot tell whether the three refer to one thing or to three.
The respondent proposes that the glossary term be used at every occurrence and that the glossary entry be linked at first use in each section rather than only at first use in the document.
The secretariat accepts this submission. The variation was stylistic and its cost to the reader exceeds any benefit.
A single term is now used throughout for each defined concept, and the glossary entry is linked at first use within each section rather than once per document.
A continuity correction has been applied to zero-event arms without being reported
The respondent submits on the draft synthesis addressing In adults with type 2 diabetes, does tirzepatide achieve greater glycaemic and weight effect than semaglutide?. This submission concerns the methods rather than the conclusion, on the view that a conclusion is only as good as the search that produced it.
Several contributing trials report no events in one arm. The pooled estimate implies that a correction was applied, since a ratio measure is otherwise undefined, but the methods do not name the correction or its magnitude, and the choice materially affects the result when events are rare.
The respondent proposes that the correction be named and that an alternative method be reported alongside for rare-event outcomes.
The secretariat accepts this submission. A correction that changes the estimate should not be invisible.
The correction is now named in the methods of every review in which one was applied, and rare-event outcomes carry a second estimate obtained by a method that does not require one.
The conclusion is stated more strongly than the certainty rating supports
This submission concerns the draft review of In adults with type 2 diabetes, does tirzepatide achieve greater glycaemic and weight effect than semaglutide?. The respondent has conducted reviews on adjacent questions and the observation arises from that experience.
The draft rates the anchor outcome at low certainty and then states the finding in the conclusion without qualification. The respondent states that the two sentences cannot both be true as written.
The respondent proposes that conclusion language be tied mechanically to the certainty rating, so that a low rating cannot produce an unqualified claim.
This point is adjacent to the one made in submission 002 and the respondent puts it in a form the secretariat can act on.
The secretariat accepts this submission. The mismatch is the failure mode the certainty framework exists to prevent.
Conclusion wording is now drawn from a fixed set of formulations tied to the certainty rating, so that a low certainty rating produces a statement that the evidence may suggest an effect and that the estimate is likely to change with further research.
Two factual descriptions of the sponsor's programme are inaccurate
The respondent’s comment on the draft review of In adults with type 2 diabetes, does tirzepatide achieve greater glycaemic and weight effect than semaglutide? arises from comparing the included set against the respondent’s own knowledge of the field.
The submission is made on behalf of the marketing-authorisation holder and is confined to two matters of fact. The draft describes a trial as terminated where the sponsor closed it at a pre-specified interim analysis, and gives a dose in a unit that does not match the approved labelling.
Supporting documentation, comprising the published trial report and the current summary of product characteristics, accompanied the submission. No view is expressed on the certainty ratings, which the sponsor considers a matter for the assessment committee.
The respondent notes submission 003 above and does not repeat the ground it covers.
The secretariat accepts this submission. Both points are matters of fact, both were checkable against documents the Institute holds, and both were wrong in the draft.
The trial status now reads as completed at a pre-specified interim analysis, with the analysis identified, and the dose is stated in the unit used in the approved labelling. The submission is identified as an industry submission on its face in accordance with the conflicts policy.
Baseline imbalance in a small contributing trial is not remarked on
The respondent read the draft review of In adults with type 2 diabetes, does tirzepatide achieve greater glycaemic and weight effect than semaglutide? and has confined this submission to a single matter.
The respondent identifies a small contributing trial with a baseline difference in the outcome variable, and states that the difference is large enough to account for a share of the reported effect.
The respondent asks that the trial be excluded or that the imbalance be addressed.
The secretariat notes this submission. The imbalance is recorded in the risk-of-bias assessment for that trial, in the domain concerning the randomisation process, where the respondent may not have looked.
No amendment arises. The judgement and the supporting figures are already reported at domain level for that study, and a sensitivity analysis omitting it is reported in the results, which does not change the direction of the estimate.
Two included studies do not meet the registered eligibility criteria
The respondent notes that the review question — In adults with type 2 diabetes, does tirzepatide achieve greater glycaemic and weight effect than semaglutide? — is answerable only if the contributing trials measured the same thing, and submits with that in view.
The respondent identifies two studies in the included set whose duration falls below the minimum stated in the protocol, and one excluded study that appears to meet every criterion.
The respondent proposes that the three be reassessed and that the outcome of the reassessment be recorded whichever way it goes.
The secretariat accepts this submission in part. On reassessment, one of the two included studies does fall below the minimum duration and has been removed. The second reports a duration that met the criterion in the protocol version registered at the time. The excluded study was excluded for a reason not clearly recorded, which was a documentation failure.
One study has been removed from the included set and the estimate recomputed, the exclusion reason for the third study has been corrected in the excluded-studies table, and the screening decisions are now recorded against the protocol version in force at the time of screening.
Subgroup findings are reported that were not registered in the protocol
Having read the draft synthesis on In adults with type 2 diabetes, does tirzepatide achieve greater glycaemic and weight effect than semaglutide?, the respondent puts one point to the committee.
The draft reports differences between subgroups that do not appear in the registered protocol. The respondent states that unregistered subgroup analysis is hypothesis-generating and that the draft presents it in the same form as the pre-specified results.
The respondent proposes that unregistered analyses be removed.
The secretariat accepts this submission in part. The analyses are retained and relabelled rather than removed, because removing an analysis that was conducted leaves no record that it was.
Every subgroup analysis is now labelled as pre-specified or post hoc against the registered protocol, post hoc analyses are reported in a separate subsection without a certainty rating, and the protocol version against which the labelling was made is stated.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.