Results & Debate

 

SciPinion engaged an independent panel of experts to serve on a science advisory panel (SAP) using methods described in Kirman et al. (2019). Roles for the review sponsor, manager, and panel members are depicted in Figure A-1. The process was designed with the goal of maximizing the pool of ideal panelists, defined as the intersection of four populations, people who have expertise in the subject matter, are objective, are available to participate, and are willing to participate. Six experts in human health exposure, toxicity, and risk assessment were identified to participate in this panel. The process for recruiting, selecting, and engaging the expert panel is described below.

 

Panel Recruitment

Potential candidates were identified as having relevant experience in mode of action, carcinogenesis (emphasis on liver), and risk assessment using a variety of sources, including: (1) SciPinion’s internal database; (2) searches for authors of recent publications on the topic of interest in online databases (e.g., Pubmed, Google Scholar); (3) searches of profiles on social media databases (e.g., LinkedIn); (4) general internet searches; and (5) referrals. Email addresses were obtained for as many potential candidates as possible.  An email invitation was sent to all potential candidates, requesting interested candidates to volunteer on https://app.scipinion.com, upload a copy of their CV, and provide a brief application statement (i.e., what makes you qualified for this panel?). SciPinion received CVs from a total of 176 applicants, 2 of which were excluded for failing to upload their CV, leaving 174 candidates to go through the next step of the process.

 

Panel Selection 

A triple blinded process was used: (1) candidates were blinded to the review sponsor; (2) the review sponsor was blinded to the candidates and played no role in selection; and (3) those selected for the panel were blinded to one another. Expertise data provided by the applicants and extracted from their CVs were used to rank the candidates with respect to general expertise metrics (e.g., academic degree, number of years of experience, number of publications) and topic-specific expertise metrics (e.g., CV key word counts). 

 

Six panel members were selected by SciPinion from the available candidates based upon the expertise metrics described above. Additional candidates were identified as potential alternates, in case a panelist is unable to complete the participation. The demographics and expertise metrics for the 6 panelists in Panel 2 are as follows:

 

·               Country of residence: Canada (2), Germany (1), United States (4)

·               Current sector of employment: Academia (3), Consulting (1), Retired/Past Government (1), Consulting or Retired/Past Industry (1)

·               Advanced degrees: PhD (5); MD (2)

·               Mean years of experience: 38±12 years

·               Mean publications: 212±58

 

Panel Engagement

The 6 panel members were placed under contract. Email addresses corresponding to their SciPinion user accounts were verified as belonging to the experts (i.e., associated with their publication record, with their place of employment, or verified by personal communication).  Charge questions were developed by SciPinion.

 

During the application process and throughout the peer review, panel members were blinded to the identities of their fellow panel members (identified online only by their display names of “Expert 1”, “Expert 2”...). Individual responses to the charge questions are linked to the experts anonymized display names, and not to their identities, an effort intended to provide psychological safety.

 

The primary review material consisted of the following summary document (see Appendix A), select references from the published literature and government reports. Panel members were also permitted to request additional publications and reports as needed to support their participation. The expert panel engagement was structured to have 3 rounds using a modified Delphi format (start in April of 2024, completion in June of 2024): 

 

·      Round 1 – Panel members worked independently to read the review material and answer Round 1 charge questions.  All 6 panel members completed their assignment as scheduled. 

·      Round 2 – Panel members worked deliberatively to review and comment on each other’s responses to Round 1 questions.  All participation was conducted online (app.scipinion.com) in an anonymous manner (i.e., experts were randomly assigned display names “Expert 1”, “Expert 2”...). A total of 39 comments were received during Round 2, with all panel members participating. 

·      Round 3 – Panel members worked independently to answer additional charge questions, including those provided by the panel during Round 1. All panel members completed this round as scheduled.

All charge questions and panel member responses from this engagement are provided below.