Draft format of the supplier dossier series — submissions
The 10 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
10 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
A printed copy loses the information that makes it checkable
This is a submission on format of the supplier dossier series, from a respondent who reads these documents on paper.
The respondent states that printing a document produces pages without the identifier, the version or the retrieval date, so that a printed copy cannot be tied back to what it was printed from.
The respondent proposes that these be printed in a fixed position on every page.
The secretariat accepts this submission. A printed assessment circulates for longer than the browser tab it came from.
The print stylesheet now carries the identifier, version, publication date and retrieval date on every page, expands link destinations into the text, and prints the full reference list rather than a link to it.
An identifier that encodes the document type will have to change when the type does
This submission concerns the draft of format of the supplier dossier series. The respondent’s interest is in retrieval: whether a document can be found again, and cited, after it has been superseded.
The respondent states that the proposed identifier embeds the series and the type, and that a document reclassified after publication would then require a new identifier, breaking every citation to it.
The respondent proposes that the identifier carry no semantic content and that type be a field rather than part of the identifier.
The secretariat accepts this submission. An identifier that has to change is not an identifier.
The identifier is now opaque and permanent from first publication, the series and type are recorded as fields that may change, and a reclassified document keeps the identifier it was issued with.
The registry is published only as pages and not in a form that can be processed
This submission addresses format of the supplier dossier series from the standpoint of a reader arriving from a search engine rather than from the front page.
The respondent, who works with the register programmatically, states that examining the document set as a whole requires extracting fields from rendered pages, and that a structured record would make the registry usable for research about the Institute itself.
The respondent proposes that the registry fields be published in a structured form alongside the pages.
The secretariat accepts this submission. A body that asks others to publish checkable records should publish one.
The registry fields are now emitted in a structured form alongside the rendered pages, carrying the identifier, series, type, version, dates, certainty rating and status for every document, and the structured record is regenerated with the site so that it cannot drift from the pages.
The citation format offered cites the document and not the part
The respondent submits on format of the supplier dossier series. A document series is a set of promises about where things will be found, and this submission concerns one of them.
A reader citing a specific section is given a citation for the whole document, so a citation cannot be checked without reading the whole of it.
The respondent proposes that the citation block offer a part-level citation carrying the section number.
The secretariat accepts this submission.
The citation block now offers a part-level citation carrying the section number and the part identifier alongside the document-level one.
A multi-part document does not indicate how many parts it has
The respondent has read format of the supplier dossier series in draft and makes one submission.
The respondent states that a reader landing on part two of a document from a search result has no indication that parts one and three exist.
The respondent proposes a parts index on every part.
The secretariat notes this submission. Every part already carries the full parts list in the margin, which the respondent may not have seen at a narrow viewport where the margin is collapsed.
No amendment to structure arises. The parts list is now rendered above the body rather than collapsed at narrow viewports, so that it is visible in the setting where the respondent found it absent.
The same quantity is presented differently in two series
The respondent notes that format of the supplier dossier series will govern a great many documents, and submits accordingly.
The respondent states that a certainty rating is displayed one way in the monograph series and another in the synthesis series, and that a reader moving between them cannot be sure the two mean the same thing.
The respondent proposes a single presentation across all series.
The secretariat accepts this submission. A difference in presentation is read as a difference in meaning.
Certainty ratings are now presented identically across every series, using the same badge, the same wording and the same accompanying formulation, and the presentation is specified in the series structure document rather than left to each series.
Quantitative claims are reproduced without the method that produced them
The respondent read format of the supplier dossier series in draft and has confined this submission to a single provision.
Several figures in the draft are quoted from sources that determined them by different methods. A figure obtained by one determination and a figure obtained by another are not comparable, and the draft places them in the same sentence without distinguishing them.
The respondent, an analytical chemist, proposes that every quantitative claim carry the method that produced it at the point of use rather than in the reference.
The secretariat accepts this submission. Placing two figures side by side is an implicit claim that they are the same kind of quantity, and in the cases identified they were not.
Every quantitative claim now carries the determination that produced it at the point of use, and figures obtained by non-comparable methods are no longer presented in the same row or sentence.
The anchor outcome for an indication is not the outcome a decision turns on
Having read the draft of format of the supplier dossier series, the respondent puts one point to the secretariat.
The respondent, a practising clinician, states that for two indications in the draft set the anchor is the outcome most often measured rather than the outcome that would determine whether to treat.
The respondent proposes that the anchors be changed and gives the alternatives.
The respondent notes submission 006 above and does not repeat the ground it covers.
The secretariat accepts this submission in part. One anchor is changed. The other is retained because no contributing trial measures the alternative, and an anchor no study reports produces a series of documents that can report nothing.
One anchor outcome is changed to the outcome proposed, and the second is retained with the alternative recorded as the preferred anchor and listed as an evidence gap, so that the reason for the compromise is on the record.
The same concept is given three different names in one document
The respondent’s comment on format of the supplier dossier series arises from consulting the series for reference rather than writing for it.
The draft refers to the same quantity as a response rate, a responder proportion and a categorical outcome in different sections. The respondent, who works in health-technology assessment, states that a reader cannot tell whether the three refer to one thing or to three.
The respondent proposes that the glossary term be used at every occurrence and that the glossary entry be linked at first use in each section rather than only at first use in the document.
The secretariat accepts this submission. The variation was stylistic and its cost to the reader exceeds any benefit.
A single term is now used throughout for each defined concept, and the glossary entry is linked at first use within each section rather than once per document.
The index shows ratings without the reasons behind them
The draft of format of the supplier dossier series was read from the standpoint of someone who will navigate the series rather than write for it.
The respondent states that a list of low ratings without their causes tells a reader that the evidence is weak but not in what respect, and that the respect is what determines whether new research would help.
The respondent proposes that the downgrade domains appear in the index row.
The respondent’s submission overlaps with submission 001 and was prepared without sight of it.
The secretariat accepts this submission. The reason is more actionable than the rating.
Every index row now carries the downgrade domains alongside the rating, and the index can be filtered by domain so that a reader can see, for example, every outcome downgraded for imprecision across the document set.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.