3.6
Share Page
Post on LinkedIn
Copy URL to Clipboard
Copied!

Results
(9 Answers)

Jump to Debate

  • Expert 3

    Consider Distribution Assumptions:

    • Adjust the calculators to account for the log-normal distribution of exposures, as the current assumption of a normal distribution may not accurately reflect real-world data.
    Address Variability:

    • Ensure that the calculators account for variability both between and within subjects, for both exposures and biomarkers.
    Incorporate Simulations:

    • Include simulations, especially since closed-form solutions may not accurately estimate sample size. Simulations, such as those using random mixed-effects models, are often necessary for precision epidemiological studies with challenging exposure assessments.
    Add Calculations for Contrast, Attenuation, and Bias:

    • Incorporate the ability to calculate contrast, attenuation, and bias, as these are critical parameters in exposure assessment and can typically be estimated using linear mixed-effects models.
    Update Justifications Based on Literature:

    • Revise the calculators to incorporate findings from established literature, such as the work by Kromhout et al. (1993) and Rappaport and Kupper's textbook on quantitative exposure assessment. This will ensure the calculators reflect current best practices in the field.
    Expand Variance Range Considerations:

    • Adjust the calculators to handle the wider range of variances and ICCs observed in studies, as suggested by Lin et al. (2005).
    Include Group Sampling Strategies:

    • Introduce group sampling strategies into the calculators, as these can optimize exposure sampling and improve study design.
    Broaden Default Values:

    • Expand the default values to include a broader range of scenarios, such as power values of 0.7 or even 0.6, and consider smaller ICCs.
    Compare Biomarker vs. Exposure Samples:

    • Provide options to evaluate whether biomarker samples or exposure samples (e.g., air samples) are more suitable for a given epidemiological study  Lin et al. (2005) .
    Shift Focus from Significance to Effect Size:

    • Emphasize the importance of effect size rather than significance alone, as significance is often a function of sample size and may not fully reflect the practical implications. e.g. having a significance range would be useful.
    Consider Time and Half-Life in Calculations:

    • Integrate considerations for the half-life of biomarkers into the calculators. This is crucial for determining the number of repeats and appropriate timing for sample collection, as highlighted by Preau et al. (2010).
    Emphasize the Importance of Pilot Studies:

    • Recommend conducting pilot studies to estimate variance components before starting full epidemiological studies. Pilot studies or simulations, such as Monte Carlo methods, can provide more accurate point estimates for parameters like significance, power, sample size, and the number of repeats needed.

  • Expert 7

     Using multiple measurements of exposure (instead of biomarkers) or at least a combination of exposure measurement when biomarker is not available

  • Expert 4

    Calculator 1:

    Sometimes the number of repeats (y-axis) has tick levels at non-integer value (e.g., at intervals of 0.5), need to fix to integer-only.

    Consider including the formula and reference in the text.

    Calculator 2:

    Default value of SD should be 1.

    The calculator is for estimating a mean, would you create another calculator for proportion estimation?

    Calculator 3a:

    I think this calculator is for the user to balance between the sample size (n) and number of measurements per subject (m), so the graph should present them. For the current setting, users need to try different values of m, obtain their corresponding n, and decide which combination to be used. The difference between n_x and n_z is not explained in the white paper.

    Calculator 3b:

    This calculator is not explained in the white paper. Can the user specify the MDE?

    Calculator 4:

    There are too few choices to set the parameters. Would be great if the users can enter the values themselves. Would you allow the users to change the number of simulations? It appears that the current simulations are too few that the bias might increase with sample size.

  • Expert 6

    1. Clarify the units of the inputs.
    2. Include more realistic (higher) OR inputs.
    3. Incorporate within-person variability in outcome measurements.
    4. The x-axis of Figure 2 should be labeled as "sample size," not "number of samples."
    5. There seem to be two Figure 4's, without clear reason.
    6. Figure 5 should make it clear that it pertains to logistic regression. 

  • Expert 1

    For calculator #4 the sensitivity analysis explorer, perhaps the odds ratio could be wider along with the sample size <100 and >800. Some people may have a limited budget which may limit the sample size and others might want to see a reasonable effect size (or: 2.0).

  • Expert 9

    As much as is practical, arrange the information and output similarly across all the calculators.

    Calculator #2:  Sample Size Calculator for Mean with Desired Margin of Error

    1. How does this relate to situations with low ICC?  Does this assume that within-individual variance is 0 or negligible?
    2. Allow standard deviation to go much higher, for example 20 or 50.  For example, the standard deviation for weight among American males is 29 pounds, and for cholesterol is 14. 
    3. Similarly, allow the desired margin of error to go much higher as well.
    4. You might consider offering a desired margin of error to be expressed as a percentage of the mean, rather than as absolute units.

    Calculator #3a and b:  Sample Size and Minimum Detectable Effect (MDE) Tradeoff Calculator for Linear Regression

    1. It seems that sometimes you use the term “within-subject variance” and other times the term “measurement error” to mean the same thing.  I suggest either clarifying the relationship between them or using the same term.
    2. It could be helpful to readers to explain how to use calculators 3a and 3b together, if that is likely to occur.  Could they be combined into a single calculator with the option to set m=1 if within-person variability is not going to be dealt with?

    Calculator #4:  Sensitivity Analysis Explorer for Logistic Regression

    1. It would be helpful to extend the Odds Ratio higher, say to 2.0 or even 3.0 (and lower, to accommodate their inverses, 0.5 and roughly 0.3).  Sample sizes needed to identify odds ratios of 1.2 as statistically significant may be so large as to be unrealistic for researchers with limited funding or dealing with exposure measurements with high variability.
    2. When I tried using the Shiny App, neither the charts nor the table changed when I changed the Sample Size using the slider.
    3. Would it be possible to change this calculator to show the necessary sample size (n X m) or n, m separately?  I think that might be more helpful to researchers designing studies.

  • Expert 8

    See previous answers. 

  • Expert 5

    Perhaps include options for including additional confounding variables (i.e. X2, X3) in the calculators.

  • Expert 2

    1. Improve the clarity of the terms used in the calculators, particularly those mentioned above.
    2. Include brief explanations or tooltips for key statistical terms in the user guides, such as ICC and validity coefficient. Additionally, provide references to relevant literature or resources for users who wish to explore these statistical concepts further. This could be done through a "Learn More" link or a short list of references in the user guide.
    3. Enhance the accuracy of labeling in the calculators to clearly indicate the units being used (e.g., “0.01 (1%)” for percentages) to help prevent any misinterpretation of input values. 

Debate (1 Comment)

Back to Top
0
Expert 7
09/13/2024 00:40
I found suggestions for improvements of Expert 3 particularly important, including
1. adding log-normal distribution of exposures
2.  Calculators should account for variability both between and within subjects, for both exposures and biomarkers
3. Ability to evaluate bias
4. Include wider range of variances and ICCs, and power values 
5. Ability to evaluate whether biomarker samples or exposure samples (e.g., air samples) or a combination of both are more suitable or available.
6. Integrate considerations for the half-life of biomarkers into the calculators to evaluate 
the number of repeats and appropriate timing for sample collection.
7. Allow for innovative designs, eg group sampling or replacement of missing biomarkers with exposure estimates
8. Move from so much focus on statistical significance
9. Importance of pilot studies to estimate parameters needed for the best design of the main study
Comments are closed for this page.