Which of the following changes in peer review do think would most improve peer review quality and timeliness and improve the quality of science being published?

Answer Explanations

• View answer

  user-287804

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	1	0
  Allow peer reviewers to review each other's review and debate each other	0	0	1	0	0
  Support double blinding in peer reviews	0	0	0	1	0
  Add quantitative measures of study quality	0	0	0	1	0
  Allow for peer review to continue after publication	0	0	1	0	0

  Once published continuous review doesn't make a difference anymore. The paper won't be taken off the journal if people don't find the results worthwhile or flawed.  
• View answer

  user-74194

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	1	0
  Allow peer reviewers to review each other's review and debate each other	0	0	0	0	1
  Support double blinding in peer reviews	0	0	1	0	0
  Add quantitative measures of study quality	0	0	0	1	0
  Allow for peer review to continue after publication	0	0	0	1	0

  I don't yet know the scope of the issue of publication failures. I used a subjective a priori approach to the choices.
• View answer

  user-504085

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	1	0	0	0
  Allow peer reviewers to review each other's review and debate each other	0	0	0	1	0
  Support double blinding in peer reviews	0	0	0	0	1
  Add quantitative measures of study quality	0	0	0	1	0
  Allow for peer review to continue after publication	0	1	0	0	0

  1. I don't agree on paying for peer-reviews as this will create an industry to make money without notable quality of the review, nor the outcome.
  2. this will take up much time and reviewers may not be inclined to further efforts. Most reviewers are academics and have little time to spare on extra-curricular activities.
  3. double blinding should work and rule out bias
  4. good idea, but how many measures would need to be taken into account?
  5. this wouldn't help, unless the post-publication comments are used to change the paper, which the journals may not agree to.
• View answer

  user-606148

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	1	0	0
  Allow peer reviewers to review each other's review and debate each other	0	0	0	1	0
  Support double blinding in peer reviews	0	0	0	1	0
  Add quantitative measures of study quality	0	0	1	0	0
  Allow for peer review to continue after publication	0	0	1	0	0

  Our experience with each of these innovations is still too limited to know the effects of adopting them at scale
• View answer

  user-426175

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	1	0
  Allow peer reviewers to review each other's review and debate each other	0	0	0	1	0
  Support double blinding in peer reviews	0	0	0	1	0
  Add quantitative measures of study quality	0	0	0	0	1
  Allow for peer review to continue after publication	1	0	0	0	0

  All these above are less important as compared to standardization, competency. Fastness is against quality.
  
  I have many more suggestions, but I do not see any intention to realize them, I do not see cooperation of large publishers. A rejected papers is not redirected to the best journal, but to the same publisher... etc.
• View answer

  user-683654

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	0	1
  Allow peer reviewers to review each other's review and debate each other	0	0	0	1	0
  Support double blinding in peer reviews	0	0	0	1	0
  Add quantitative measures of study quality	0	0	0	1	0
  Allow for peer review to continue after publication	0	1	0	0	0

  It is good if peer reviewers are paid. 
• View answer

  user-730223

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	1	0
  Allow peer reviewers to review each other's review and debate each other	1	0	0	0	0
  Support double blinding in peer reviews	0	1	0	0	0
  Add quantitative measures of study quality	0	0	0	1	0
  Allow for peer review to continue after publication	0	0	0	1	0

  Often the peer reviewer may not be an expert in all the areas being written in the manuscript.  May only focus on areas of familiarity.
• View answer

  user-214133

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	0	1
  Allow peer reviewers to review each other's review and debate each other	1	0	0	0	0
  Support double blinding in peer reviews	0	0	0	0	1
  Add quantitative measures of study quality	0	0	0	1	0
  Allow for peer review to continue after publication	0	0	0	1	0

  The interaction between authors and reviewers needs to be anonymous.
• View answer

  user-640046

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	1	0
  Allow peer reviewers to review each other's review and debate each other	0	0	1	0	0
  Support double blinding in peer reviews	0	0	0	1	0
  Add quantitative measures of study quality	0	0	1	0	0
  Allow for peer review to continue after publication	0	1	0	0	0

  i disagree with continued review after publication. What is the purpose? To change the paper after published?
• View answer

  user-286232

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	0	1
  Allow peer reviewers to review each other's review and debate each other	0	0	0	0	1
  Support double blinding in peer reviews	0	0	0	0	1
  Add quantitative measures of study quality	0	0	0	0	1
  Allow for peer review to continue after publication	0	0	0	0	1

  As already explained in previous section. Post publication reviews help in corrective course of action, especially applied research.
• View answer

  user-774853

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	1	0
  Allow peer reviewers to review each other's review and debate each other	0	0	0	0	1
  Support double blinding in peer reviews	0	1	0	0	0
  Add quantitative measures of study quality	0	1	0	0	0
  Allow for peer review to continue after publication	0	0	0	0	1

  see responses above
• View answer

  user-771708

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	1	0	0
  Allow peer reviewers to review each other's review and debate each other	0	0	1	0	0
  Support double blinding in peer reviews	0	1	0	0	0
  Add quantitative measures of study quality	1	0	0	0	0
  Allow for peer review to continue after publication	0	0	1	0	0

  peer review already continues after publication in the public domain
  
  quantitative measures of study quality is a mirage
• View answer

  user-41956

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	1	0
  Allow peer reviewers to review each other's review and debate each other	0	0	1	0	0
  Support double blinding in peer reviews	0	0	1	0	0
  Add quantitative measures of study quality	0	0	0	0	1
  Allow for peer review to continue after publication	0	0	1	0	0

  I am intrigued by the possibilities of designing and using structured review formats. Especially ones that would speed up the process for the reviewer, and provide more uniform reviewer input. Adding a quantitative measures could be part of such efforts.
• View answer

  user-70793

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	0	1
  Allow peer reviewers to review each other's review and debate each other	1	0	0	0	0
  Support double blinding in peer reviews	0	0	1	0	0
  Add quantitative measures of study quality	0	0	1	0	0
  Allow for peer review to continue after publication	0	1	0	0	0

  The only change I am confident about changing the peer-review process is the payment of the peer reviewers.
• View answer

  user-767467

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	1	0	0	0
  Allow peer reviewers to review each other's review and debate each other	0	0	1	0	0
  Support double blinding in peer reviews	0	0	0	1	0
  Add quantitative measures of study quality	0	0	0	1	0
  Allow for peer review to continue after publication	0	0	0	0	1

  Once published there is usually no ongoing review. Some published articles are known to be untrue later but never get corrected.
• View answer

  user-337244

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	0	1
  Allow peer reviewers to review each other's review and debate each other	1	0	0	0	0
  Support double blinding in peer reviews	0	0	0	0	1
  Add quantitative measures of study quality	0	0	0	0	1
  Allow for peer review to continue after publication	0	1	0	0	0

  Allow peer reviewers to review each other's review and debate each other will be more contradicting the research and also break the blind process rule. So, it will no a best idea. Allow for peer review to continue after publication 
• View answer

  user-135268

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	0	1
  Allow peer reviewers to review each other's review and debate each other	1	0	0	0	0
  Support double blinding in peer reviews	1	0	0	0	0
  Add quantitative measures of study quality	0	0	0	1	0
  Allow for peer review to continue after publication	1	0	0	0	0

  There is nothing more pointless than allowing peer review to continue after publication. It's just ridiculous. We didn't shelve Newtonian physics because Einstein's understanding of physics is accepted today. What are you going to do!? Take Isaac Newton's papers out of print because Einstein is alive?
  
  Adding quantitative measures of study quality provides a standard, facilitates the evaluation process and ensures that the material being evaluated is evaluated accurately. In this regard, journals can use specially trained AIs to get a preliminary assessment for quantitative measurement, and the AI can score according to a scoring system. The reviewer can then use this as a template and use the AI's opinions in their evaluation. A fully automated AI assessment is not immediately feasible today, but it can start now to assist the reviewer in the assessment.
  
  There is nothing more absurd than allowing peer assessors to review and discuss each other's assessments. An evaluation that lasts forever!? Of course not.
  
  
  
• View answer

  user-521833

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	1	0	0	0
  Allow peer reviewers to review each other's review and debate each other	0	0	0	0	1
  Support double blinding in peer reviews	0	0	0	0	1
  Add quantitative measures of study quality	0	0	0	1	0
  Allow for peer review to continue after publication	0	0	0	0	1

   Paying reviewers may be a double-edged sword if not handled with care. While it can incentivize timely and thorough reviews, it also risks turning peer review into a transactional exercise, where the focus shifts from quality to quantity. Without proper guardrails and accountability measures, the integrity of the review process could be compromised, and superficial or poorly reasoned evaluations may become the norm. 
   
  Adding quantitative measures to assess study quality can be helpful, but only if the metrics are clearly defined and systematically applied. It’s not enough to simply ask reviewers to rate a paper on a 0–100% scale without context. Instead, journals need to adopt structured scoring frameworks that break down the evaluation into key components, for example, methodological rigor, originality, clarity, reproducibility, and relevance to the field. This approach promotes consistency, transparency, and fairness, and provides both authors and editors with clearer guidance on how the manuscript meets (or falls short of) scientific standards. 
  
  One valuable but often missing element in most peer-reviewed journals—many of which I have reviewed for or served on editorial boards—is the opportunity for peer reviewers to review and critique each other’s evaluations. Allowing reviewers to engage in constructive debate could significantly improve the quality of the review process and help editors who are not subject matter specialist make the right calls. It would help surface stronger reasoning, expose weak or unfounded critiques, and ensure that good and useful science is not unfairly dismissed, while also guarding against the publication of spurious claims. Ultimately, this kind of transparent reviewer dialogue could enhance scientific rigor and public trust. 
  
  I would advocate for double blind or double transparency depending what's at stake. Double-blind review helps reduce bias and ensures objectivity, while double transparency promotes accountability, professionalism, and trust. Adopting elements of both, depending on context, can strengthen the integrity and effectiveness of the peer-review process. 
  
  Peer review should not end at publication—that's precisely why journals offer a platform for post-publication comments and critiques. These should be taken seriously by editors and journals, as they provide a vital mechanism for ongoing scientific scrutiny and refinement. While it's understood that publishing is also a business, the credibility and integrity of science must remain the top priority. Encouraging and responding to post-publication peer review is essential for correcting errors, clarifying uncertainties, and ensuring that published research continues to evolve with new insights. 
• View answer

  user-881641

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	0	1
  Allow peer reviewers to review each other's review and debate each other	0	0	0	1	0
  Support double blinding in peer reviews	0	0	0	0	1
  Add quantitative measures of study quality	0	0	0	0	1
  Allow for peer review to continue after publication	0	1	0	0	0

  1. Incentivizes Quality and Accountability
  
  

  Right now, peer review is often unpaid, done on top of already overloaded academic schedules. Paying reviewers:
  
  

  • Encourages deeper, more thoughtful reviews, because there's a tangible value tied to the effort.
    
    
  • Makes reviewers more accountable—if someone’s getting compensated, they’re more likely to take the job seriously and meet deadlines.

  2. Speeds Up the Review Process
  
  

  One of the biggest complaints in academic publishing is how slow peer review can be. Paying reviewers:
  
  

  • Provides a real reason to prioritize the review.
    
    
  • Makes it easier for journals to set and enforce deadlines, especially if payment is conditional on timely submission.
    
    

  3. Supports Equity in Academia
  
  

  Right now, only those with the luxury of time and institutional support can afford to do unpaid reviewing. Paying reviewers:
  
  

  • Helps early-career researchers, adjuncts, and scholars from underfunded institutions participate.
    
    
  • Makes the system more inclusive and representative of a broader range of voices.

  4. Professionalizes the Process
  
  

  Reviewing is a crucial part of scholarly publishing. Paying for it:
  
  

  • Acknowledges it as a professional service, like editing or teaching.
    
    
  • Helps shift the culture to one that values and respects peer review as essential academic labor.
    
    

  5. Encourages Participation
  
  

  Many researchers are overwhelmed and burnt out. By offering compensation:
  
  

  • Journals can attract more reviewers and avoid overburdening the same small group.
    
    
  • It might help reduce reviewer fatigue, a growing issue in many fields.
    
    
• View answer

  user-219698

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	1	0	0	0	0
  Allow peer reviewers to review each other's review and debate each other	0	1	0	0	0
  Support double blinding in peer reviews	1	0	0	0	0
  Add quantitative measures of study quality	1	0	0	0	0
  Allow for peer review to continue after publication	1	0	0	0	0

  As above
• View answer

  user-441980

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	0	1
  Allow peer reviewers to review each other's review and debate each other	0	1	0	0	0
  Support double blinding in peer reviews	0	1	0	0	0
  Add quantitative measures of study quality	0	1	0	0	0
  Allow for peer review to continue after publication	0	1	0	0	0

  Introduce a performance-based compensation model for reviewers and editors.
  
  

  • Pay reviewers and editors for their time and expertise, recognizing that peer review is skilled labor.
    
    
  • Compensation should be tied to quality, not just completion — evaluated by editors and Editors-in-Chief (EICs) based on timeliness, depth, and constructiveness of the reviews.
    
    
  • Create a scoring system to track reviewer/editor performance over time, helping journals maintain a pool of reliable, high-quality contributors.
    
    

  Benefits:
  
  

  • Encourages more timely and thoughtful reviews.
    
    
  • Rewards dedicated reviewers who consistently deliver value.
    
    
  • Raises the overall quality of published science by emphasizing rigor and accountability.
    
    
  • Helps build a more sustainable and professional peer review ecosystem.
    
    
• View answer

  user-526097

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	0	1
  Allow peer reviewers to review each other's review and debate each other	1	0	0	0	0
  Support double blinding in peer reviews	0	0	0	1	0
  Add quantitative measures of study quality	0	0	0	0	1
  Allow for peer review to continue after publication	1	0	0	0	0

  We don't need to create conflict, we need to create a situation where valuable publications actually get published.
• View answer

  user-332757

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	0	1
  Allow peer reviewers to review each other's review and debate each other	0	0	0	1	0
  Support double blinding in peer reviews	0	0	0	1	0
  Add quantitative measures of study quality	0	1	0	0	0
  Allow for peer review to continue after publication	0	1	0	0	0

   Based on all my answers, I don't think I agree with allowing peer review to continue after publication, as it would add even more workload for the reviewers; without benefit por the publication.
• View answer

  user-82487

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	1	0	0	0	0
  Allow peer reviewers to review each other's review and debate each other	1	0	0	0	0
  Support double blinding in peer reviews	0	0	0	0	1
  Add quantitative measures of study quality	0	1	0	0	0
  Allow for peer review to continue after publication	1	0	0	0	0

  No author would ever look at the comments after the article has been published. It will be a totally sterile excercise; waste of reviewer's time.
• View answer

  user-573537

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	1	0	0	0
  Allow peer reviewers to review each other's review and debate each other	0	0	1	0	0
  Support double blinding in peer reviews	0	0	0	0	1
  Add quantitative measures of study quality	0	0	0	0	1
  Allow for peer review to continue after publication	0	0	0	0	1

  However, one must recognize this is not an easy task.
• View answer

  user-650721

  	Terrible idea	Might not help	Unclear	Would help somewhat	Would help tremendously
  Pay peer reviewers	0	0	0	1	0
  Allow peer reviewers to review each other's review and debate each other	0	1	0	0	0
  Support double blinding in peer reviews	0	0	0	1	0
  Add quantitative measures of study quality	0	0	0	0	1
  Allow for peer review to continue after publication	0	0	1	0	0

  I addressed these in question above. This is essentially same question twice…