This SciPinion panel (SciPi 646) was assembled to help develop best practices for the use of biomonitoring data for exposure assessments in epidemiology studies, with a particular focus on addressing within- and between-individual variability in biomonitoring data, following the established SciPinion methods described in Kirman et al. (2019). The process was designed with the goal of maximizing the pool of ideal panelists, defined as the intersection of four populations, people who have expertise in the subject matter, are objective, are available to participate, and are willing to participate. Nine experts in Biomonitoring, Epidemiology, and Statistics were identified to participate in this panel. The process for recruiting, selecting, and engaging the expert panel is described below.
Panel Recruitment
Potential candidates were identified as having relevant experience in epidemiology, biomonitoring, exposure assessment and statistics using a variety of sources, including: (1) SciPinion’s internal database; (2) searches for authors of recent publications on the topic of interest in online databases (e.g., Pubmed, Google Scholar); (3) searches of profiles on social media databases (e.g., LinkedIn); (4) general internet searches; and (5) referrals. Email addresses were obtained for as many potential candidates as possible. An email invitation was sent to all potential candidates, requesting interested candidates to volunteer on https://app.scipinion.com, upload a copy of their CV, and provide a brief application statement (i.e., what makes you qualified for this panel?). SciPinion received applications from a total of 257 applicants, 19 of which were excluded for failing to upload their CV, leaving 238 candidates to go through the next step of the process.
Potential candidates were identified as having relevant experience in epidemiology, biomonitoring, exposure assessment and statistics using a variety of sources, including: (1) SciPinion’s internal database; (2) searches for authors of recent publications on the topic of interest in online databases (e.g., Pubmed, Google Scholar); (3) searches of profiles on social media databases (e.g., LinkedIn); (4) general internet searches; and (5) referrals. Email addresses were obtained for as many potential candidates as possible. An email invitation was sent to all potential candidates, requesting interested candidates to volunteer on https://app.scipinion.com, upload a copy of their CV, and provide a brief application statement (i.e., what makes you qualified for this panel?). SciPinion received applications from a total of 257 applicants, 19 of which were excluded for failing to upload their CV, leaving 238 candidates to go through the next step of the process.
Panel Selection
Expertise data provided by the applicants and extracted from their CVs were used to rank the candidates with respect to general expertise metrics (e.g., academic degree, number of years of experience, number of publications) and topic-specific expertise metrics (e.g., CV key word counts).Nine panel members were selected by SciPinion from the available candidates based upon the expertise metrics described above. Additional candidates were identified as potential alternates, in case a panelist was unable to complete the participation. The demographics and expertise metrics for the 9 panelists in the panel are as follows:
Expertise data provided by the applicants and extracted from their CVs were used to rank the candidates with respect to general expertise metrics (e.g., academic degree, number of years of experience, number of publications) and topic-specific expertise metrics (e.g., CV key word counts).Nine panel members were selected by SciPinion from the available candidates based upon the expertise metrics described above. Additional candidates were identified as potential alternates, in case a panelist was unable to complete the participation. The demographics and expertise metrics for the 9 panelists in the panel are as follows:
· Country of residence: Australia (1), Sweden (1), Canada (3), United Kingdom (1), United States (3)
· Current sector of employment: Academia (7), Retired/Past Government (2)
· Advanced degrees: PhD (8); MD (1)
· Mean years of experience: 27.8
· Mean publications: 252.4
Panel Engagement
The 9 panel members were placed under contract. Email addresses corresponding to their SciPinion user accounts were verified as belonging to the experts (i.e., associated with their publication record, with their place of employment, or verified by personal communication). Charge questions were developed by SciPinion. A triple blinded process was used: (1) candidates were blinded to the review sponsor; (2) the review sponsor was blinded to the candidates and played no role in selection; and (3) those selected for the panel were blinded to one another. During the application process and throughout the peer review, panel members were blinded to the identities of their fellow panel members (identified online only by their display names of “Expert 1”, “Expert 2”...). Individual responses to the charge questions are linked to the experts' anonymized display names, and not to their identities, an effort intended to provide psychological safety.
The 9 panel members were placed under contract. Email addresses corresponding to their SciPinion user accounts were verified as belonging to the experts (i.e., associated with their publication record, with their place of employment, or verified by personal communication). Charge questions were developed by SciPinion. A triple blinded process was used: (1) candidates were blinded to the review sponsor; (2) the review sponsor was blinded to the candidates and played no role in selection; and (3) those selected for the panel were blinded to one another. During the application process and throughout the peer review, panel members were blinded to the identities of their fellow panel members (identified online only by their display names of “Expert 1”, “Expert 2”...). Individual responses to the charge questions are linked to the experts' anonymized display names, and not to their identities, an effort intended to provide psychological safety.
The experts were charged with reviewing a white paper on the topic (written by two independent experts (project leads) who were also blinded to the sponsor) as well as review and vet several statistical calculators developed by the project leads. The basis of these calculators has been summarized in the primary review material (White Paper). Optional review materials included the primary literature relied on for the equations used in the calculators. The calculators used are linked below:
· Calculator #1: Desired Validity Coefficient: https://scipinion.shinyapps.io/RepeatsForDVC/
· Calculator #2: Sample Size for MOE: https://scipinion.shinyapps.io/SampleSizeMOE/
· Calculator #3a: Linear regression N: https://scipinion.shinyapps.io/LinearRegressionN/
· Calculator #3b: Linear regression M: https://scipinion.shinyapps.io/LinearRegressionM/
· Calculator #4: Sensitivity Analysis Explorer: https://scipinion.shinyapps.io/SensitivityAnalysisExplorer/
Panel members were also permitted to request additional publications and reports as needed to support their participation.The expert panel engagement was structured to have 3 rounds using a modified Delphi format (start in August of 2024, completion in September of 2024):
· Round 1 – Panel members worked independently to read the review material and answer Round 1 charge questions. All 9 panel members completed their assignment as scheduled.
· Round 2 – Panel members worked deliberately to review and comment on each other’s responses to Round 1 questions. All participation was conducted online (app.scipinion.com) in an anonymous manner (i.e., experts were randomly assigned display names “Expert 1”, “Expert 2”...). A total of 43 comments were received during Round 2, with all panel members participating.
· Round 3 – Panel members worked independently to answer additional charge questions, including those provided by the panel during Round 1. All panel members completed this round as scheduled. All charge questions and panel member responses from this engagement are provided below. (edited)