Improving Epidemiology Study Designs When Using Biomonitoring for Exposure Assessment

Within-person variability is almost accounted for in studies with more than one observations from a person (e.g., repeated measures studies), but between-person variability is often ignored, partly due to the lack of historical data in ICC of the study population, and the other part due to lack of statistical tools for study design.

Good discussion, I think many points have been addressed already. Some notes from my side: 
-Part of the discrepancy in our original responses indeed seems to stem from a more semantic issue, from different interpretations of 'consider'. Maybe we can clarify this in the next round to improve our discussion.
-Expert 3 noted a difference in approach between European and US-based research groups. It would be interesting to explore this further, e.g.; is there a difference in regional guidelines or established practices? Or is this disparity more field-specific; are European groups more active in specific fields where variability consideration is more common? In general, this suggests a need for more collaboration and harmonization of approaches. 
-Expert 1 and others already mentioned the crucial trade-offs between sample size and repeated measures. This balance is a key topic in this discussion, as resources are always constraint. 

Please note that I answered the question whether the calculators were "user-friendly" not if they were extremely useful.  I think there was an error in the response categories.

It seems from the various expert comments, that the consideration of within-subject variation is very dependent on the application area (Expert 1), the expertise of the research group (and here I would say that the Europeans are more advanced in their approach), and the background of the Principal Scientists initiating the work.  I personally find that individuals trained in exposure assessment/science or industrial hygiene, and those who have real-world field experience collecting exposure (air, dermal etc.) and biological monitoring samples will consider all of the possible sources of variation (including variation in chemical analysis results) that could affect epidemiological study power and risk estimates. 

The process of conducting biomonitoring studies with adequate Power (sample size) and repeated measures is a balancing act. Usually there are trade offs (i.e., larger sample size and less repeated measures or smaller sample size and more repeated measures). In addition, the application area  (e.g., Forensic, Clinical, Sports Medicine, Environmental) will also affect these factors and the metabolite/chemical being measured and the technique plays a role. I agree with Expert 2 that the within variation is not usually addressed but again this depends on the purpose for biomonitoring. If I was measuring a chemical/motabolite in an athlete within variation would be critical over time.   

I agree with Expert 6. 

 Expert 2 has well summarized the issues: actual practice is hindered by logistical and educational barriers. Expert 5 has pointed out an interesting aspect that I have observed but had not considered before—the "within" variation is not as commonly addressed as between-subject variation. I would add that I have seen this both among epidemiologists and toxicologists. It is more common to have more subjects than to take repeated measurements. I wonder if there is another barrier, in addition to education, logistics, and finances, such as cognitive bias or a preference for simpler study designs. 

I agree that Expert 3 provides valuable insights on improving these statistical tools. However, we must also consider the balance between realism and idealism. Highly complicated tools are not easily adopted by a broader audience, especially those with limited statistical and epidemiological knowledge. In practical situations, for example in sample size or power calculations, we do not expect to give an exact value. Instead, providing an approximate range that informs users of potential uncertainties in their estimations is more helpful. As an experienced statistician, I believe that a simplified, suboptimal, but more user-friendly solution would be more popular, without significantly compromising scientific integrity.

Although I recognize the awareness and application of statistical methods to account for between- and within-person variability in epidemiology studies, as Expert 3 points out, the application of such practices is far from universal. Often, these practices are restricted to well-resourced groups or those with specific expertise. This raises concerns about the overall reliability of epidemiological findings, especially those from under-resourced settings or non-expert groups. Moreover, the practical challenges of incorporating such variability—highlighted by the need for extensive measurements and additional resources—suggest that its regular consideration might be more idealistic than realistic for many studies. This underlines a significant divide: while the theoretical importance of considering variability is clear, the actual practice is hindered by logistical and educational barriers. This situation underscores the need for more standardized practices, better resource allocation, and accessible tools in the epidemiological community.

Agree to Expert 3's comments about the issues of the calculator. The authors could consider adopting these assumptions / parameter settings to their calculator.

In response to Expert 6:  It looks like Expert 3 gave two responses.

Regarding Question 1.1:  I suspect the variability in our responses might be due at least partly to different interpretations of the broad phrasing of the question.  For example, I chose to interpret "consider" generally as "think about" and not rigorously as "include directly when doing sample size calculations".  Furthermore and as others have mentioned, I believe most epidemiologists use between-person variability in sample size calculations as it is usually required.  Within-person variability seems to be less commonly included in those calculations in my experience.  And consideration of both types of variability is limited by factors such as availability of data, and/or not be described in detail in the Methods section of published articles.

Why is the number of answers (and number of experts) = 9, while the graph shows 10 responses?

Within-person variability is usually accounted for and has a larger magnitude than between-person variability, but still between-person variability should be accounted for, as it can cause an underestimation of the sample size required.

As we judge that these calculators and sample size calculations in general mostly useful,  it might be helpful to discuss limitations of such calculations.  Is anyone aware of a comparison between sample size calculations and the resulting sample size of studies (due to missing data, resource limitations etc.)

There seems to be a consensus that these calculators are mostly useful for researchers less experienced in power calculations and that those with more expertise will likely require/prefer more flexibility for more complex data. 

- There indeed seems to be a significant divide in how half-life is considered. I like expert 3's observation about  categorizing responses into toxicology/pharmacology versus epidemiology.  However, given that many epidemiologists have backgrounds in other health-related fields (which could include toxicology), and e.g. toxicologists apply epidemiological methods, hopefully the divide is not as stark as it might appear. 
- As with our previous question, it would be useful to specify/clarify what we mean by 'consider' for our further discussion. 
-Expert 7 provides an interesting comment how the half-life (e.g. shorter versus longer) is important in determining how important within-person variability is. It would be good to take this into account in our discussions.
-  While direct comparisons are unlikely, I'm curious if anyone can share experiences where consideration of half-life significantly influenced study design, results or conclusions vs studies that did not? E.g. re-analyses of existing data or datasets that were enriched with additional measurements?

Good comments, I have nothing else to add here.

These responses indicate to me that the calculators may be most useful for researchers not well trained in these concepts, as expert 2 says - to reach a broader audience.  

I would add that we always encourage our students/colleagues to consult with a statistician/epidemiologist (especially if they are clinical/MDs) prior to submitting any grant application/initiating research/and ongoing.  Calculators may be handy for sample size and power calculations, but more advanced expertise is usually needed.

I look at this issue from both sides of the fence - I'm an epidemiologist who also was trained/conducted research in exposure assessment, biological monitoring and toxicology.  Early in my career I realized that most PhD level epidemiologists had little understanding/training in exposure assessment/biological monitoring. The only discipline at that time that took these issues into consideration (and more specifically biological half-lives) was in nutritional epidemiology.  

Now, we include lectures on these issues (toxicokinetics/toxicodynamics etc.) in graduate level courses generally in occupational/environmental/dietary epidemiology or exposure measurement,  but I believe that the majority of epidemiologists "out there" have minimal understanding. 

 I agree with Expert 8's suggestion to put the calculators on GitHub. The calculators could be presented at various levels of complexity, with options gradually expanded. 

 Expert 8 has very thoughtfully identified the issue. It seems to me that the expert opinions on this panel may be categorized into two groups: Group A, consisting of toxicologists and pharmacologists who actively work with biomarkers and consider half-lives essential in their research, and Group B, comprising epidemiologists who may not work with half-lives as regularly. Maybe Group B may be more accustomed to working with biomarkers that have longer half-lives, which could simplify the statistical analysis in epidemiological studies, even if the concept of half-life is not explicitly addressed. 

In consideration of the thoughtful comments by my colleagues here, I believe I interpreted the word "consider" too broadly, and will revise my response downward in Round 3.

I believe the target audience for these tools is not statisticians or epidemiologists, as there are already many complex and specialized packages available to address these issues. Instead, these tools are designed for those with less expertise in power or sample size calculation, but who are still aware of the variability within and between individuals. As I previously mentioned, simplified and user-friendly tools are needed to reach a broader audience.

The experts' opinions clearly diverge, highlighting variations in disciplinary focus and the specific requirements of different subfields. For example, as Expert 8 points out, those in toxicology and pharmacology are more likely to consider the half-life of biomarkers due to the direct relevance of pharmacokinetics in their work. In contrast, researchers in broader fields such as environmental epidemiology may not prioritize or possess the necessary expertise to incorporate these specific pharmacokinetic considerations into their studies.

I strongly agree with Expert 7's comment about biomarkers of longer-term exposure.

Expert 5's comment about writing own programs for sample size estimation is a reality. If the authors' targets are epidemiologists / statisticians, they need to improve the flexibility of the calculator.

 It seems the question was interpreted somewhat differently by the different experts. Rereading it, I would also interpret it as referring to numerical magnitudes of variability. But expert 3 also makes an interesting point that ratios may be more important than absolute values. 

The between variation is zero here. ;-)

Expert 3 makes a good point about the ratio of between/within variation (generally the ICC-intraclass correlation).

The experts' answers on this panel demonstrate a strong understanding of between- and within-person variability, as well as the advanced statistical analyses used to assess these variabilities. Based on the comments, it seems that some experts may not focus as much on the exact numerical values (e.g., Expert 9). Perhaps the question was somewhat ambiguous, and experts may not have explicitly addressed it.  In my work, there also seems to be a greater emphasis on the ratio of between- to within-person variability rather than the absolute values.  The ratio is often more important than the absolute values—e.g., when within-person variability exceeds between-person variability, it suggests the need for more repeats instead of more subjects. This could be similar to recognizing the significance of a study without necessarily recalling the exact p-value. 

 I agree. The settings for the parameters in the tools could be more flexible. For example, allowing adjustable effect sizes and offering optional relative risk measures such as hazard ratios, risk ratios, or incidence rate ratios would enhance their utility. 

Some experts raised the importance of expanding the possible range of OR beyond 1.2 which make sense to me. I am also looking forward to extending the calculator for other effect size measures, e.g., relative risk (expert 3) and hazard ratio (expert 8)

While I think these calculators are likely to improve the design of studies, they may also be useful in other areas, such as teaching students about different sources of variability in a study and their impact on study design.

 It seems that the assumptions need further explanation and should be more clearly stated. 

Expert 6 raises a very good point about the importance of discussing the need for repeat measurements. I believe it would be beneficial to expand the discussion further and incorporate the aspects mentioned by Expert 2. Additionally, I wonder if including a visual could be particularly useful to show various ICCs. 

 I'm pleased to observe that the experts' opinions are largely consistent. 

Agree with Expert 2's comment on the use of statistical terms, especially different fields have their own terms. Suggest to provide some background information about the calculator.

I found suggestions for improvements of Expert 3 particularly important, including
1. adding log-normal distribution of exposures
2.  Calculators should account for variability both between and within subjects, for both exposures and biomarkers
3. Ability to evaluate bias
4. Include wider range of variances and ICCs, and power values 
5. Ability to evaluate whether biomarker samples or exposure samples (e.g., air samples) or a combination of both are more suitable or available.
6. Integrate considerations for the half-life of biomarkers into the calculators to evaluate 
the number of repeats and appropriate timing for sample collection.
7. Allow for innovative designs, eg group sampling or replacement of missing biomarkers with exposure estimates

8. Move from so much focus on statistical significance
9. Importance of pilot studies to estimate parameters needed for the best design of the main study

Explicitly addressing likely missing data

I suggest greater discussion of options beyond increase in number of exposure measurements per individual or increase in number of individuals when intra-individual exposure variability is high.  Standardizing timing of samples during the day or by season is mentioned, but a simpler approach is to compare outcomes in groups of individuals known to have very high vs very low exposures by virtue of their occupations, residence, etc.

 I really appreciate Expert 9's suggestions and would like to expand on the ideas provided in Expert 9's discussion comment. Could the calculators offer direct, actionable recommendations or additional outputs beyond just sample size? For example, could they suggest increasing between-individual variability as a recommendation? 

 Expert 4 has an excellent idea: include costs. 

About Expert 1's comments on sample size calculator for RCTs, I am not aware of any researchers that incorporate measurement error when estimating the sample size for RCT, so more work has to be done among trial statisticians before this could happen.