Independent · non-commercial · publishes on a quarterly cycle|Current cycle 2026 Q3
Compound Evidence InstituteEvidence synthesis · established 2023Graded assessments of compounds, trials, methods and supply
Document set current to 30 July 2026
Public comment period · §3

Draft supplier evaluation rubric — disposition

The disposition of every submission and the amendments that resulted.

Document identifier
CEI-CP-002/3
Series
Public comment period
Version
1.0
Published
22 Oct 2023
Last reviewed
22 Oct 2023
Next review
22 Oct 2024
Identifier
10.71829/cei.cp.2
Certainty
Not rated
Cycle
2023 Q3
Window
06 Aug 2023 – 20 Sep 2023
Status
Closed
Submissions
10

§3Disposition and amendments

§3.1Disposition table

Table 1. Every submission with its disposition. Each row links to the submission in full.

ReferenceRespondentPoint raisedDisposition
DRAFT-SUPPLIER-RUB/001Dr Vasilisa ImmelmannTwo criteria can be satisfied without changing anything a supplier doesAccepted
DRAFT-SUPPLIER-RUB/002Georgiana ZimmerthalDeclared interests should appear on the document rather than on a separate pageNoted, no amendment
DRAFT-SUPPLIER-RUB/003Dr Quentin ZimmerthalTwo anchors on the same criterion can both describe the same supplierAccepted in part
DRAFT-SUPPLIER-RUB/004Dr Fitzwilliam Danquah-ÖbergDocumentary practice is weighted equally with analytical evidenceAccepted in part
DRAFT-SUPPLIER-RUB/005Dr Eamon ImmelmannWhether a purchaser can obtain the record behind a figure is not scoredAccepted
DRAFT-SUPPLIER-RUB/006Stellan Cholmondeley-AdeScores from different rubric versions are presented togetherAccepted
DRAFT-SUPPLIER-RUB/007Dr Yaa KettlewellPoint estimates are given without an intervalAccepted in part
DRAFT-SUPPLIER-RUB/008Dr Zebedee Zaleski-MbekiAn accreditation claim is scored without being verifiedAccepted
DRAFT-SUPPLIER-RUB/009Dr Oisín RautavaaraScores are reported to a precision the underlying judgements do not supportAccepted in part
DRAFT-SUPPLIER-RUB/010Dr Frideswide NordhagenA high score invites a purchasing decision the rubric cannot supportAccepted
10 submissions in total.

§3.2Summary by disposition

Table 2. Counts by disposition, with the meaning of each.

DispositionCountMeaning
Accepted5The submission is accepted and the draft is amended as proposed. The amendment is recorded in the amendment log of the document it changed and is traceable to the numbered submission.
Accepted in part4Part of the submission is accepted and part is not. The secretariat response states which part is which and on what ground the remainder was declined.
Noted, no amendment1The submission raises a point the Institute accepts but that does not require a change to the draft, most often because the draft already states it elsewhere. The location is given in the response.
Not accepted0The submission is not accepted. The secretariat response gives the reason. A submission that is not accepted remains published in full; the Institute does not remove a submission because it disagrees with it.

§3.3Resulting amendments

  1. Two criteria can be satisfied without changing anything a supplier does — arising from DRAFT-SUPPLIER-RUB/001. Both criteria are restated so that they are satisfied only by a document evidencing the underlying activity, such as a dated determination or a signed release record, and the rubric now states for every criterion what evidence satisfies it.
  2. Two anchors on the same criterion can both describe the same supplier — arising from DRAFT-SUPPLIER-RUB/003. The anchors are retained in prose, because a checklist would score a certificate carrying every field badly filled as though it were complete. Each anchor now names the fields that must be present for it to be reached, so that two anchors cannot describe one…
  3. Documentary practice is weighted equally with analytical evidence — arising from DRAFT-SUPPLIER-RUB/004. Analytical evidence now carries greater weight than documentary practice, and a supplier that does not identify the laboratory performing its determinations cannot reach the upper band regardless of its total score.
  4. Whether a purchaser can obtain the record behind a figure is not scored — arising from DRAFT-SUPPLIER-RUB/005. Availability of the underlying record on written request is now an anchor within the transparency criterion, tested by request during each assessment cycle and recorded in the correspondence log.
  5. Scores from different rubric versions are presented together — arising from DRAFT-SUPPLIER-RUB/006. Every score now carries the rubric version and the assessment date, the comparison table displays only scores made under the current version, and dossiers not yet reassessed are shown as awaiting reassessment rather than with a stale score.
  6. Point estimates are given without an interval — arising from DRAFT-SUPPLIER-RUB/007. Every estimate now carries its interval where the source reported one, and where it did not, the estimate is annotated as reported without an interval rather than left to appear as a precise figure.
  7. An accreditation claim is scored without being verified — arising from DRAFT-SUPPLIER-RUB/008. Credit for accreditation is now awarded only where the Institute has verified the accreditation and its scope against the accrediting body's register, the verification date is recorded in the dossier, and an unverifiable claim is recorded as claimed and…
  8. Scores are reported to a precision the underlying judgements do not support — arising from DRAFT-SUPPLIER-RUB/009. Scores are now reported to the precision the criteria support, the criterion-level judgements are published alongside the total so that the derivation can be checked, and the dossier states that a difference smaller than the reported precision is not a…
  9. A high score invites a purchasing decision the rubric cannot support — arising from DRAFT-SUPPLIER-RUB/010. The scope of the score is now printed in the same row as the score on every surface that carries it, in the same weight, rather than in a separate notice.

References cited on this page

References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.

  1. International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute

Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.

Nothing published by the Institute is medical advice, a diagnosis, a prescription, a treatment recommendation or a purchasing recommendation. Compounds supplied for research use are not approved for human or veterinary use in any jurisdiction, and a favourable analytical assessment of a supplier is not a statement that any product is safe or effective. The Institute publishes certainty ratings and never recommendations. No telephone number, messaging handle or ordering channel appears anywhere on this site.