SciPi 417: Peer review of a publication on polymer biodegradation and emissions
Please rate your confidence in the Rolsky and Kelkar (2022) based on consideration of the following study components (1=lowest confidence; 10=highest confidence). Please explain your rating in the space below.
Results
Confidence Rating (n) | Total | |
---|---|---|
Conceptual model (pathways and compartments) | 8 | |
Model assumptions and equations (e.g., steady-state mass balance) | 7 | |
Model predictions for polyvinyl alcohol degradation | 8 | |
Model predictions for polyvinyl alcohol in post-treatment wastewater | 7 | |
Sensitivity Analyses | 7 | |
Uncertainty Analyses | 7 | |
Author Conclusions: Are the conclusions drawn supportable based on the methods used and results reported? Are there alternate conclusions that could/should be presented? | 8 |
Answer Explanations
- Expert 8
Component Confidence Rating Conceptual model (pathways and compartments) 4 Model assumptions and equations (e.g., steady-state mass balance) 2 Model predictions for polyvinyl alcohol degradation 3 Model predictions for polyvinyl alcohol in post-treatment wastewater 3 Sensitivity Analyses 1 Uncertainty Analyses 1 Author Conclusions: Are the conclusions drawn supportable based on the methods used and results reported? Are there alternate conclusions that could/should be presented? 2 The conclusions drawn are not supported by the data. No good conclusions can be drawn due to the lack of available relevant data.
According to this modeling methodology, emissions from large cities are overestimated.
Data on figure 4 appears to be based on wrong assumptions. The amount of untreated PVA and the amount remaining in biosolids are probably much lower. I would not base any policy decision on this data. - Expert 5
Component Confidence Rating Conceptual model (pathways and compartments) 8 Model assumptions and equations (e.g., steady-state mass balance) 8 Model predictions for polyvinyl alcohol degradation 7 Model predictions for polyvinyl alcohol in post-treatment wastewater 6 Sensitivity Analyses 5 Uncertainty Analyses 4 Author Conclusions: Are the conclusions drawn supportable based on the methods used and results reported? Are there alternate conclusions that could/should be presented? 4 The overall model is well designed; I am unsure of the exact WWTP flows used (see above). The breakdown of PVA is based on published studies; though one would expect given the wide use of PVA in pods that most WWTP bacteria should be somewhat adapted to the presence of PVA by now. Other loss processes are likely, but not quantified (skimming of oil; loss of PVA in aeration tanks). Given that only "average" WWTPs are considered, those with AOP would have a much stronger impact on PVD removal.
- Expert 6
Component Confidence Rating Conceptual model (pathways and compartments) 9 Model assumptions and equations (e.g., steady-state mass balance) 3 Model predictions for polyvinyl alcohol degradation 4 Model predictions for polyvinyl alcohol in post-treatment wastewater 4 Sensitivity Analyses 3 Uncertainty Analyses 5 Author Conclusions: Are the conclusions drawn supportable based on the methods used and results reported? Are there alternate conclusions that could/should be presented? 4 Conceptual model (evaluated based on Figure 2):
The conceptual model includes all generally accepted pathways and the standard compartments expected in the average US WWTP. A potential improvement would be the inclusion of a kind of sink compartment for the leakage of sewage into the environment outside of the treatment cycle. This could combine sanitary outflows, runoff events, and expected leakage. Personally, I would have preferred a 'hypothetical' compartment where the efficiency etc. of more modern (but not unconventional) treatments such as AOP would have been highlighted. This would help to translate the findings to geographical regions (but within and outside the US), and may also serve to maintain relevance of this article when these advanced processes become more standardized in the coming decades.Model assumptions (evaluated based on equations (1) to (12) and related text):
Several minor and a few major points of concern. Generally, the formulas make sense and could be used as the basis of a robust model. However, the critique below is likely to stack and severely impacts my rating of the resulting model.
Equation (1) and associated text does not take into account the fraction of water used in ways that do not end up in the sewage system. This could be e.g., watering the garden or industrial processes where water is part of the product as either solvent or reactant (concrete). This means the volume may be overestimated.
Equation (2) calculates the total wastewater generated assuming 20 % losses. This 20 % is based on one reference which are the proceedings of a workshop regarding sewage facilities in INDIA. At the very least, one would expect an explanation why these proceedings from India are usable to estimate the losses in US sewage systems. In several aspects, India is still considered a developing country and sewage handling is likely to be one of these aspects. Therefore, it would be logical to assume that leakage and other losses of sewage would be much larger in India than in a well-developed country such as the US.
Equation (3) is based on a calculation error. The total US wastewater treatment is 34 BGD divided over 16,000 WWTPs. The authors then calculate this to be 2.12 BGD per facility, which is a factor 1,000 too high. It should be 2.12 million gallons per day (MGD). The value of ~2 MGD is much more in line with what is normally considered to be the normal capacity of a WWTP. I expect the authors actually used the correct quantity in their calculations, but the fact that the manuscript was published with such a gross error does impact my confidence.
Equations (4) to (6) are probably the main part of the modeling attempt where theoretical consideration to apply a general model to the entire US conflict with the (theoretical) robustness of the model. It was assumed in Equation (3) that each WWTP in the US has the same capacity. In reality, it is more likely that populous states have larger WWTPs with much higher capacities than the plants one would expect in the rural states. Comparing Figure 3 with a map color-coded based on the population per state has striking similarities. This means there is a high likelihood that there is a significant source of bias/confounding introduced by Equations (4) to (6). The authors do not address this as a source of bias. In fact, the word "bias" or "confound-" is not used once in the entire paper, which is quite perplexing as composition of sewage is well-known to be dependent on population density as well as socioeconomic factors (e.g., https://pubs.acs.org/doi/10.1021/acs.estlett.0c00392).
Equation (7) is based on the online survey the authors conducted to investigate frequency and type of detergent purchased, which has just 527 respondents (less than 11 respondents per state, on average). Considering the size and geographical as well as cultural diversity within the US, the sample size is extremely small to be extrapolated to the entire US. In fact, this number would probably be considered difficult to extrapolate for a small city, let alone an entire state or the entire US. The online survey is also a potential source of selection bias, as an online survey is more likely to be filled in by people that are technological savvy and it could be argued such people more readily fit the demographic that is more likely to use LDPs. In addition to this, the inclusion of just three brands of laundry pods and two brands of dish pods seems to be a bit meager, especially considering the high RSD in the average weight of the laundry pods. There is a factor four difference between the upper and lower end of the range; based on this alone the amount of PVA stemming from this source could be over- or underestimated by a factor two.
Equation (8) to (10) build upon the previous equations and thus suffer from the same potential issues undermining their trustworthiness.
Equation (11) and (12) also build upon the previous equations. However, Equation (11) is likely to suffer at least some confounding/bias in both the Mpva as well as the WWut parameter. Depending on the specific US state, these two sources of confounding could partially cancel each other out, but they may also be in the same direction thus increasing the significance of confounding/bias. Regardless of the combined confounding effect, these sources of confounding should have been identified and addressed in the manuscript.Model predictions for polyvinyl alcohol degradation (evaluated based on reviewing relevant abstracts of articles listed in the Supp Inf, focused on articles with DOI):
Articles that use a much higher PVA concentration than expected in wastewater: 5, 7, 8, 19, 29, 31.
Article 6 states that an adapted culture can biodegrade PVA with a <30 h lag phase and in a single stage.
Article 10 and 11 appear to be a solid source of support for the biodegradation of PVA in WWTPs.
Article 21 provides a solid source of support for the biodegradation of PVA under anaerobic conditions in WWTPs and the general environment.
The authors do not elaborate in much detail about how they translated data from screening-type and lab-size studies to full-scale WWTP, but it appears that their main approach is to directly translate results in small-scale screening studies to the WWTP-scale. It is questionable whether this is reasonable and the authors should have backed this decision up with literature. While the translation of experimental data to the WWTP-scale is complex and complicated, it is a vital pivot point in their assessment and as such should be backed up firmly by existing literature. In my opinion, the authors should have included a confidence/prediction interval or estimated range with their biodegradation estimates (i.e., the green values in Figure 2 and associated values in the text). Especially so, considering the fact that there is also well-documented variability between WWTP designs, combined with day-to-day as well as seasonal variability in WWTP efficiency.Model predictions for polyvinyl alcohol in post-treatment wastewater (evaluated based on the evaluation of the previous two aspects and text within the manuscript):
Since the model assumptions, equations, and the author's translation of lab-based data to WWTP-scale are all flawed to a certain extent, the predictions for PVA in effluent have a similar (low) level of confidence. Furthermore, the author's predictions are not backed up with environmental monitoring data of PVA. It might also have been informative if the predicted effluent output would have been compared with other well-known (and preferably related) polymers.Sensitivity Analyses (evaluated based on manuscript text and the above evaluations on model confidence ratings):
The authors do not specifically mention any kind of sensitivity analysis in their manuscript. In the evaluation above, several parameters have been highlighted that may lead to significant uncertainty reduction if they would have been further optimized. In my opinion, the most significant parameters are:
- Online survey with a very low number of respondents was used to determine use of LDPs over the entire US. It is not possible (and perhaps would be far-fetched) to perform a Power calculation, but I expect that for a reasonably reliable result there would have been at least 10,000-20,000 responses. Furthermore, it makes sense to have the online survey evaluated per state, since the Mpva and WWut/WWt are also reported per state, and it is not unreasonable to expect significant differences between states in the use (and amounts used) of LDPs. The online survey may also introduce selection bias, since the demographic most likely to respond may also be more likely to use LDPs.
- There is very high variability in the average weight calculated for LDPs, especially for the laundry pods. The authors included only three brands of laundry pods and three replicates per brand. With such high variability (1.0 +/- 0.6) it would make sense to go for more replicates per brand and also increase the number of brands. The authors ask numerous questions about preferred brands in their online survey, but it is not clear how this information was used to determine which brands to use for the weight determination. However, it is unlikely that three main brands could be identified from such a survey, since the number of (premium) brands across the US is probably a factor 10-20 higher than 3.
- There is a source of bias/confounding in the determination of WWTP capacity per state. The authors assume each WWTP in the US has the same capacity, while it is more likely that the populous states have a significant number of much larger WWTPs while the rural states may have a relatively large number of relatively small WWTPs. It should be straightforward to obtain reports from several states regarding the WWTP capacity and/or effluent volumes. The authors could have compared such data for at least 20-25 % of the states to support the outcome of their modeling approach.
- The parameter for leakage of sewage appears to be based on data from India, although I was not able to review the workshop proceedings. However, 20 % leakage seems unrealistically high for a developed country like the US. While this error is not multiplied or otherwise enhanced by further modeling steps, it is one of the input variables that directly impacts the output variables. It should have been better supported and the authors should have explained how their reference is relevant for the US and/or which translational steps they performed to make the value suited for the US.Uncertainty Analyses (evaluated based on manuscript text and the above evaluations on model confidence ratings):
The authors do not specifically mention any kind of uncertainty analysis in their manuscript. In the evaluation above, several parameters have been highlighted that may lead to significant uncertainty reduction if they would have been further optimized. The authors do include standard deviations and/or expected ranges for some of the input and some of the output parameters, but it is not always clear where these values come from. Some of the most important parameters (biodegradation estimates within different compartments of the WWTP) do not list any uncertainty ranges, while these parameters have been constructed from combining dozens of values from scientific literature. In addition, the ranges the authors do report in output parameters seem quite narrow considering the layers of uncertainty in the different equations and the WWTP model itself. Overall, I feel that the authors could have done a much better job to quantify and assess the uncertainty of their approach.Author Conclusions (evaluated based on manuscript text). Firstly, it should be noted that the last chapter is not necessarily presented as a conclusion. It does include concluding statements, so I have evaluated this chapter as being the main conclusion. Several remarks in the last chapter are not backed up by literature references e.g.,:
- The pathways of sludge have been well documented, as has the ability of WWTPs to act as sources of contaminants and microplastics entering the environment. These emissions can cause deleterious impacts on surrounding ecosystems and the biota within them. PVA that passes through conventional water treatment can similarly pose a threat to the environment in several ways, once released into the environment or if land applied.
- As the sorption of organic and inorganic pollutants is not limited to hydrophobic compounds but can also occur with hydrophilic compounds, PVA could act as a vector for transport up the food chain, similarly to more conventional plastics.
- Prior research has demonstrated that WWTPs are sources of microplastic pollution in natural and built environments. This is due to the fact that microplastics in treated sludge, termed “biosolids”, can have a variety of harmful effects on ecosystems beyond contaminant adsorption.
General assessment of the conclusion:
I feel that the last chapter is focused primarily on highlighting (additional) literature references that could support the detrimental environmental implications of PVA release into the environment. The results of the extensive modeling efforts are only briefly mentioned and no real recommendations are made to decrease environmental implications or otherwise improve the risk identified (or perceived) by the authors. Also, the authors do not put their findings with respect to PVA from LDPs in the broader context of total PVA production/emissions, or compare it with other (biodegradable) plastics. This hinders the ability of the reader to quickly assess the relative importance/urgency of the topic addressed. - Expert 1
Component Confidence Rating Conceptual model (pathways and compartments) 5 Model assumptions and equations (e.g., steady-state mass balance) 4 Model predictions for polyvinyl alcohol degradation 3 Model predictions for polyvinyl alcohol in post-treatment wastewater 3 Sensitivity Analyses 1 Uncertainty Analyses 1 Author Conclusions: Are the conclusions drawn supportable based on the methods used and results reported? Are there alternate conclusions that could/should be presented? 4 - Conceptual model (pathways and compartments) confidence rating (1 – 10)
Ans: 5 – The authors has provided a conceptual model for mass balance in Figure 2, however the reliability of the model/pathway is unclear. The conceptual model of this paper is generally clear and easy to understand. The authors first explained the mass balance of PVA in a conventional activated sludge treatment plant (Figure 2) using a simple roadmap consisting of all essential elements. This figure gives a rapid insight to the reader on the fate of PVA when subjecting into the treatment plant. Then, equation 1 until 12 were introduced to explain the calculation mechanism of the PVA estimation before treatment until post-treatment (Figure 4).- Model assumptions and equations (e.g., steady-state mass balance) confidence rating (1 – 10)
Ans: 4 – The authors have provided some equations, but the reliability is unclear since the data/findings seems insufficient and unclear. Methods and equations were not properly defined and explained.- Model predictions for polyvinyl alcohol degradation confidence rating (1 – 10)
Ans: 3 – This paper lacks data/figure/graph that can prove the reliability of the model predictions. Also, the PVA model shown in Fig 4 is unclear. How the data were obtained and measured? For example, how the environmental release is measured? What are the environmental component/parameters used in measuring the environmental release?- Model predictions for polyvinyl alcohol in post-treatment wastewater confidence rating (1 – 10)
Ans: 3 – This paper lacks data/figure/graph that can prove the reliability of the model predictions. Methods and equations were not properly defined and explained. How the PVA in post-treatment wastewater is measured? More information needed.- Sensitivity Analyses confidence rating (1 – 10)
Ans: 1 – Such analyses cannot be found in the paper- Uncertainty Analyses confidence rating (1 – 10)
Ans: 1 – Such analyses cannot be found in the paper- Author Conclusions: Are the conclusions drawn supportable based on the methods used and results reported? Are there alternate conclusions that could/should be presented? confidence rating (1 – 10)
Ans: 4 - There is no conclusion section in this paper. However, I found a paragraph that summarize the study at the last paragraph of Section 6 – Implications. However, the summary is too general, vague and does not reflect the findings from the study.- Please explain your answers
I have slightly lower confidence towards the proposed equations/models. The reason is because the authors applied the outdated data on water use and application in the United States published in 2015. Whether the data used is still representable now or the time when the authors performing this study remained questionable. In addition, whether the water use and application could be affected by the covid-19 pandemic should be considered. Overall, the final estimate might be overestimated or underestimated.
The mass balance done to predict the PVA emissions in WWTP and percentage of PVA biodegraded in each WWTP unit operation was entirely based on the assumptions done by Garrido et al. in 2013 [reference 54]. The authors could have gained more conclusive results if actual samplings of waste water in WWTP were done in year 2020 onwards. This would have eliminated the data uncertainty that comes with assumptions and they would have more confidence in reporting the degree of degraded PVA in each section.
Also, it is hard to imagine how a sand filter (c.f. Figure 2) in a WWTP that will have a high possibility to trap all the solid matters or lipids/fats content could have 0% of PVA in it. Based on operational experience, the trapped solid matters, fats or lipid content could be rich in PVA and It is perhaps too hasty to conclude that this filter will likely not have a significant effect on PVA concentration unless there is real experiment data to support this.
- Expert 7
Component Confidence Rating Conceptual model (pathways and compartments) 8 Model assumptions and equations (e.g., steady-state mass balance) Model predictions for polyvinyl alcohol degradation 3 Model predictions for polyvinyl alcohol in post-treatment wastewater Sensitivity Analyses Uncertainty Analyses Author Conclusions: Are the conclusions drawn supportable based on the methods used and results reported? Are there alternate conclusions that could/should be presented? 1 I primarily focused on parameter estimation using biodegradation data (my expertise) and therefore did not rate all components
- Expert 2
Component Confidence Rating Conceptual model (pathways and compartments) 8 Model assumptions and equations (e.g., steady-state mass balance) 8 Model predictions for polyvinyl alcohol degradation 6 Model predictions for polyvinyl alcohol in post-treatment wastewater 5 Sensitivity Analyses 1 Uncertainty Analyses 1 Author Conclusions: Are the conclusions drawn supportable based on the methods used and results reported? Are there alternate conclusions that could/should be presented? 6 The model is quite clearly laid out, and the assumptions are mostly OK (but see comments above).
The predictions are only as good as the data on degradation taken from the literature. This appears to have been quite well abstracted, but there is clearly considerable variability in parameter estimates across studies, so there is some leeway for interpretation in coming up with 'best' estimates that represent the average conditions in a US WWTP. Once the residual PVA enters receiving waterbodies, there is even greater uncertainty, but that is due to lack of data rather than any inherent problem with the paper.
Where the paper is quite weak is the sensitivity/uncertainty analysis, which is largely non-existent (see weakness 2 above) - Expert 4
Component Confidence Rating Conceptual model (pathways and compartments) 8 Model assumptions and equations (e.g., steady-state mass balance) 7 Model predictions for polyvinyl alcohol degradation 7 Model predictions for polyvinyl alcohol in post-treatment wastewater 6 Sensitivity Analyses 4 Uncertainty Analyses 4 Author Conclusions: Are the conclusions drawn supportable based on the methods used and results reported? Are there alternate conclusions that could/should be presented? 5 The integrative modeling approach is a strength, but the weaknesses in the data fed into the model, so, I have relatively high level of confidence in the conceptual model and the general equations, which are appropriately stated. I did not see specific mistakes in the equations. I do fault the approach in not conducting sufficient sensitivity and uncertainty analyses. The assumption that "most" wastewater treatment facilities in the U.S. are old is too broad, and instead, the authors could have conducted focused case studies on selected methods to represent the best-case and worst-case scenarios. The online survey is not representative.
- Expert 3
Component Confidence Rating Conceptual model (pathways and compartments) 8 Model assumptions and equations (e.g., steady-state mass balance) 5 Model predictions for polyvinyl alcohol degradation 2 Model predictions for polyvinyl alcohol in post-treatment wastewater 2 Sensitivity Analyses 5 Uncertainty Analyses 5 Author Conclusions: Are the conclusions drawn supportable based on the methods used and results reported? Are there alternate conclusions that could/should be presented? 4 As already identified as one of the key drawbacks of the model: it cannot be ruled out that the extent of biodegradation in a WWTP is underestimated. This would not scrutinize the model in itself, but it would underestimate the predicted emissions in the environment. Although a sensitivity analysis is lacking, it is highly likely that given the hydrophilicity of PVA, biodegradation is the key process affecting environmental inputs.