Draft synthesis: Dependence of reported purity on the… — submissions
The 4 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
4 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
The review protocol is described but its registration record is not linked
The draft review of How much does a reported chromatographic purity figure vary with the method used to obtain it? was read against the certificates and reports it draws on.
The respondent asks for the prospective registration record of the review protocol so that the registered outcomes can be compared with the reported ones.
The respondent states that without it the claim of prospective registration cannot be checked.
The secretariat accepts this submission in part. The protocol is published in full on the Institute site with its version history, which permits the comparison the respondent asks for. Where an external registration identifier is not held by the Institute, the review says so rather than supplying one.
Each synthesis now links its own protocol with the version in force at screening, and states explicitly where an external registration identifier is not held, so that the absence is a recorded fact.
Results obtained by different methods are pooled without recording the method
The respondent has read the draft synthesis on How much does a reported chromatographic purity figure vary with the method used to obtain it? and makes one submission about the analytical evidence it pools.
The contributing results were obtained by more than one determination, and the draft reports a single distribution.
The respondent states that the determinations do not measure the same quantity, that the difference between them is systematic rather than random, and that a pooled distribution across methods has no referent.
The respondent proposes that every result carry the method that produced it and that pooling occur only within a method.
The secretariat accepts this submission. A figure whose method is not recorded cannot be compared with another figure, and the draft pooled across the difference.
Every contributing result now carries its determination, pooling occurs only within a determination, and where a survey did not report the method the result is listed as method not stated and excluded from the pooled estimates.
Multiple vials from one lot are counted as independent observations
The respondent notes that the review question — How much does a reported chromatographic purity figure vary with the method used to obtain it? — is answerable only if the contributing determinations measured the same quantity, and submits with that in view.
Several contributing surveys report results for more than one vial drawn from a single lot, and the draft counts each vial as a separate observation.
The respondent states that vials from one lot share a fill and a manufacturing history, so the effective number of independent observations is the number of lots and not the number of vials, and that the interval reported is narrower than the data support.
The respondent proposes that the unit of analysis be the lot, with within-lot variation reported separately.
The secretariat accepts the unit-of-analysis point and declines to discard the within-lot data, which are informative about fill uniformity.
The unit of analysis is now the lot throughout, the number of lots is reported beside the number of vials, and within-lot variation is reported as a separate quantity rather than folded into the pooled interval.
Regulatory assessment documents were not searched
The respondent’s comment on the draft review of How much does a reported chromatographic purity figure vary with the method used to obtain it? arises from comparing the pooled figures against the reports they were taken from.
The respondent states that regulatory assessment reports frequently contain analyses that never appear in journals, and that a search limited to bibliographic databases will miss them.
The respondent proposes that regulatory documents be searched for every review in the series.
The respondent has read submission 002 with interest and adds one observation the secretariat may find useful.
The secretariat accepts this submission in part. Regulatory assessment documents are searched. The proposal that they be treated as equivalent to a full study report is declined, because the level of detail varies and is often insufficient for risk-of-bias assessment.
Published regulatory assessment documents are now searched as a named source, are reported as a distinct evidence class in the included-studies table, and contribute to the assessment while being excluded from pooled estimates where risk of bias could not be assessed from the document.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.