4
SciPoll 647: Misinterpretation of Results in Scientific Publishing
When science results are misinterpreted by the authors of a paper, what are the most common reasons? (check all that apply)
Results
(138 Answers)
Answer Explanations
- Confirmation bias Cognitive bias Context/importance of results are misinterpreted/inflateduser-887652Again: My field exists primarily to confirm its biases.
- Inadequate expertise in statistics Correlation is mistaken for causation Context/importance of results are misinterpreted/inflateduser-536777We need more statistic information. Furthermore, it is necessary experience in order to not to fail in our work as reviewers.
- Cognitive biasuser-876062Authors may fail to adequately acknowledge the limitations of their study, leading to overconfidence in their results and misinterpretation of the broader applicability or significance of their findings.
- Confirmation bias Cognitive bias Inadequate expertise in statistics Letting statistical significance trump critical thinking Context/importance of results are misinterpreted/inflateduser-444481I selected the following reasons because:
a) Confirmation and cognitive biases: These are innate human instincts that are not very well altered even by experienced researchers. A lot of workers may unintentionally pursue those interpretations that they would like to see supported – either they are testing their hypothesis or adopting their prior knowledge. b) Inadequate expertise in statistics: Whereas the methods of the research have been enhanced, few researchers have been able to master advanced statistics to analyze data leaving most analysis with undue interpretations. c) Letting statistical significance trump critical thinking: The emphasis on the p-values at less than 0. 05 can make the researchers exaggerate the importance of the results that even though they are statistically significant, they can be mere ridicules in the real sense. d) Context/importance of results are misinterpreted/inflated: Publishing findings is competitive due to the availability of countless publications hence authors tend to exaggerate the uniqueness or the significance of the findings. - Confirmation bias Cognitive bias Faulty experimental design Inadequate expertise in statistics Letting statistical significance trump critical thinking Correlation is mistaken for causation Context/importance of results are misinterpreted/inflated Otheruser-837221other = inadequoate knowledge of old work, work by marginal groups or published in low citation journals. in short, a biased education
- Cognitive bias Faulty experimental design Inadequate expertise in statistics Letting statistical significance trump critical thinking Correlation is mistaken for causationuser-655473In many cases of research, the supervisors force the researchers to present a wrong interpretation of the results. And researchers are often forced to report the same wrong interpretation (for example: statistical significance instead of critical thinking) in order to get the necessary points.
- Cognitive biasuser-589243A lot of people have the stupid belief that the more published they are, the more renowned and good doctors they are.
This is absurd. And untrue. - Confirmation bias Cognitive bias Faulty experimental design Inadequate expertise in statistics Letting statistical significance trump critical thinking Correlation is mistaken for causationuser-219504All these factors in some or other, lead to misinterpretation
- Inadequate expertise in statistics Letting statistical significance trump critical thinking Context/importance of results are misinterpreted/inflateduser-35552Well explained in the above sections
- Inadequate expertise in statistics Letting statistical significance trump critical thinking Correlation is mistaken for causation Context/importance of results are misinterpreted/inflateduser-937607In general, I would say "letting statistical methodology trump critical thinking" -- it's not just statistical "significance". We give a tremendous amount of authority over to statistical models with little real understanding or acknowledgement of the underlying assumptions and what sources of uncertainty are quantified vs. unquantified. Our scientific culture highly values "objective"-seeming statistical methods without adequate interrogation.
- Otheruser-156962All of the above
- Cognitive bias Inadequate expertise in statisticsuser-496176
To conduct experiments with positive results, the researcher must have some experience in this field of knowledge, conduct practical experiments step by step, and have deep cognitive (not superficial) thinking.
- Inadequate expertise in statisticsuser-457926Misinterpretation of scientific results is often due to inadequate expertise in statistics, which involves misunderstanding statistical methods, incorrect testing, and potential biases. Other factors like cognitive biases and faulty experimental design also contribute to misinterpretation.
- Faulty experimental design Inadequate expertise in statistics Letting statistical significance trump critical thinkinguser-683654The above choices are likely the best reasons.
- Inadequate expertise in statistics Otheruser-678105Lower quality of people receiving PhDs today due to reduced concern for excellence at most universities.
- Confirmation bias Cognitive bias Faulty experimental design Letting statistical significance trump critical thinkinguser-935064It seems to me that there is less emphasis on critical thinking in all levels of education. Not enough vetting of potential research question is done before launching into the study. Once in a study, there are pressures to see it through to some form of publication that often is not sufficiently meritorious.
- Inadequate expertise in statisticsuser-525512Incorrect statistical analysis or misinterpretation of statistical data can lead to erroneous conclusions.
- Confirmation bias Letting statistical significance trump critical thinking Correlation is mistaken for causation Context/importance of results are misinterpreted/inflateduser-957551At R1 institutions, tenure track faculty will not survive nor thrive without funding and generation of papers (not just one or two per year). Thus, the bias for publishing positive data (meaning always finding an effect in a toxicology experiment [see above for my broad definition of what kinds of studies are included under toxicology]) is a big driver because the "chemical apocalypse" is the "squeaky wheel that gets the grease". The bias is facilitated by p-hacking (noted under "letting statistical significance trump critical thinking") and conflation of correlation and causation. The context/importance of the results is often pushed by ignoring what the actual environmental exposures are, or worse, the idea that any exposure to anything is 'bad' (again, ignoring how physiology is based on fundamental biochemical kinetics phenomena and disbelief in a threshold concept). I always love the final sentence in published papers..."we need further study", meaning, I hope to get more grant money. I'm still an active faculty member after 46 years, but perhaps my cynicism is a behavioral biomarker that should make me retire. No, I'm just a skeptic who used to have a lot of colleagues with similar thoughts, but now most have retired.
- Inadequate expertise in statistics Correlation is mistaken for causation Context/importance of results are misinterpreted/inflateduser-130453In connection with the inadequate esxpertise in statistics, study authors only think about statistics when the raw data have already been obtained when it should have already been incorporated at the onset of experiment design.
- Correlation is mistaken for causationuser-577239Sometimes statistics significance is much more important than clinical significance
- Confirmation bias Cognitive bias Faulty experimental design Inadequate expertise in statistics Letting statistical significance trump critical thinking Correlation is mistaken for causation Context/importance of results are misinterpreted/inflated Otheruser-803407All the ticked reasons may contribute to misinterpretation, plus lack of experience.
- Faulty experimental design Inadequate expertise in statistics Correlation is mistaken for causation Context/importance of results are misinterpreted/inflated Otheruser-532952Other : deliberate cheating. In South Africa there have been some cases where academics were found to have cheated and were dismissed as University Staff. There are occasionally also post-grad students who falsify results. This also leads to expulsion.
- Cognitive bias Faulty experimental design Inadequate expertise in statistics Letting statistical significance trump critical thinkinguser-987379It is a sum of causes that can lead to authors minsinterpreting their own work.
- Letting statistical significance trump critical thinking Correlation is mistaken for causation Context/importance of results are misinterpreted/inflated Otheruser-480186Under the time pressure, some authors are quick to believe numbers rather than logic. Also, a sort of marketing thinking is spreading among scientists and this leads to inflation of the importance of results. Everybody wants to be "visible" and important.
In toxicology and risk/hazard assessment conservative approach may easily lead to overestimation of the risk/hazard. E.g., although 25 years ago Bruce Ames warned against misinterpretation of animal carcinogenicity studies when carcinogenicity was observed at exposures exceeding MTD, it is still common practice to take such results as a bases for risk/hazard assessment. - Cognitive bias Faulty experimental design Letting statistical significance trump critical thinking Context/importance of results are misinterpreted/inflateduser-512616I am particularly concerned by mistakenly applied methods and statistical tests.
- Faulty experimental design Inadequate expertise in statistics Context/importance of results are misinterpreted/inflateduser-49529As I said previously, flaws in the designs of experimental models can lead to erroneous results and another concern may be not having good statistical support.
Not everyone has all the resources at their disposal to be able to investigate, despite the conditions the researcher always works in search of fulfilling his dreams. - Confirmation bias Faulty experimental design Inadequate expertise in statistics Letting statistical significance trump critical thinking Context/importance of results are misinterpreted/inflated Otheruser-320876Misinterpretations of data often arise from intense propaganda and biases by certain members of the medical community, who may seek to sow confusion for personal gain and career longevity within publicly funded initiatives. The reductionist approach in science, which is predicated on flawed assumptions, leads to biased interpretations of data that do not serve the true progress of science. In my studies, I have characterized the outcomes of such flawed experiments, which consistently fail and let down the public, as 'medical/scientific Ponzi schemes'. Reflecting on the motives that drive misguided scientific directions and data misinterpretations, Saul Bellow aptly observed, "A great deal of intelligence can be invested in ignorance when the need for illusion is deep."
- Faulty experimental design Inadequate expertise in statistics Correlation is mistaken for causationuser-583550Most importantly evidence based decision will be highly affected.
- Inadequate expertise in statisticsuser-841416Mos are lacking or have inadequate knowledge in statistics, hence analysis and interpretation of the results is a challenging piece hence even the results are impaired
- Confirmation bias Context/importance of results are misinterpreted/inflateduser-555078Sometimes the results are misinterpreted due to imagination by the authors
- Confirmation bias Inadequate expertise in statistics Letting statistical significance trump critical thinking Correlation is mistaken for causation Context/importance of results are misinterpreted/inflateduser-58000With all the possible explanations, it can be stated that biasness regarding the result confirmation, inadequate statistical competency and the over-exacerbation of the statistical significance have a great impact on the article generalization in real-world scenario. These factors leads to diversion of the study actual results to different path which might have great impact.