[ABE-L] Fwd: {MEDSTATS} Re: "Basic and Applied Psychology" Bans The Use of P Values and Confidence Intervals

[Basilio De Braganca Pereira]

Heriberth
Este email pode ser relevante para seu seminário e para a discussão



Enviado do meu iPhone

Início da mensagem encaminhada

> De: "Paul Barrett" <paul em pbarrett.net>
> Data: 8 de março de 2015 16:54:34 BRT
> Para: <medstats em googlegroups.com>
> Assunto: RE: {MEDSTATS} Re: "Basic and Applied Psychology" Bans The Use  of P Values and Confidence Intervals
> Responder A: medstats em googlegroups.com
> 
> Martin
>  
> It’s no good ‘nit-picking’ like this.
>  
> The reason why so many articles on this issue decry NHST is because the bulk of researchers in psychology (at least) do implicitly ‘accept’ the research hypothesis. In the same vein of misunderstanding, so many researchers have no idea how to interpret confidence intervals (e.g. Hoekstra, R., Morey, R.D., Rouder, J.N., & Wagenmakers, E-J. (2014). Robust misinterpretation of confidence intervals. Psychonomic Bulletin and Review (in press; DOI 10.3758/s13423-013-0572-3),  1-8.
>  
> At the heart of this issue, right here, right now, is the relevance of NHST and statistical inference at all to scientific investigation; that is, the detection of phenomena and the understanding of their cause.
>  
> When the teaching of NHST by statisticians and methodologists to psychologists and medical investigators has appeared to fail so miserably, for so long, one has to question why, and whether NHST actually serves any scientific purpose. Precisely what Charles Lambdin did in 2012.
>  
> Lambdin, C. (2012). Significance tests as sorcery: Science is empirical - significance tests are not. Theory and Psychology, 22, 1, 67-90.
> Abstract
> Since the 1930s, many of our top methodologists have argued that significance tests are not conducive to science. Bakan (1966) believed that “everyone knows this” and that we slavishly lean on the crutch of significance testing because, if we didn’t, much of psychology would simply fall apart. If he was right, then significance testing is tantamount to psychology’s “dirty little secret.” This paper will revisit and summarize the arguments of those who have been trying to tell us—for more than 70 years—that p values are not empirical. If these arguments are sound, then the continuing popularity of significance tests in our peer-reviewed journals is at best embarrassing and at worst intellectually dishonest.
>  
> From p. 73-74 …
> “           In their defense of significance testing, Mulaik et al. (1997) argue that such misconceptions about NHST are irrelevant as to whether we should continue its use. This is an awkward position. The point that Mulaik et al. seem to be missing is that methodologists are not trying to refute NHST when they write of how it is commonly misconstrued. The case made in this context is that the very concept of a p value is so seldom accurately grasped that the deleterious impact on the quality of research of which the social sciences are comprised is undoubtedly great—so much so that it should be quite uncontroversial to state that most published research is in fact nonsense. The great philosopher of science Imre Lakatos considered most published research in the social sciences to be little more than “intellectual pollution” (reported by Meehl, 1990) and Ioannidis (2005) argues that the spread of the p value from psychology to other fields has resulted in a world where, even in medicine, most published findings are probably wrong.
>             Mulaik et al.’s (1997) point is still a sound observation. A proper response, however, is not to continue with our blind use of NHST, but to do something about it. Before proceeding, let us state just what exactly a p value tells us. A p value is the probability of obtaining the results in hand, assuming that the statistical null hypothesis is true in the population. That is all and nothing more. As Schopenhauer (1851/2004) reminds us, nothing more is implied by a premise than what is already contained in it, and this, it is time we admit, does not imply much.”  
>  
> And Marc Branch in 2014:
> Branch, M. (2014). Malignant side effects of null-hypothesis significance testing. Theory and Psychology, 24, 2, 256-277.
> Abstract
> Six decades-worth of published information has shown irrefutably that null-hypothesis significance tests (NHSTs) provide no information about the reliability of research outcomes. Nevertheless, they are still the core of editorial decision-making in Psychology. Two reasons appear to contribute to the continuing practice. One, survey information suggests that a majority of psychological researchers incorrectly believe that p values provide information about reliability of results. Two, a position sometimes taken is that using them to make decisions has been essentially benign. The mistaken belief has been pointed out many times, so it is briefly covered because of the apparent persistence of the misunderstanding. The idea that NHSTs have been benign is challenged by seven “side-effects” that continue to retard effective development of psychological science. The article concludes with both a few suggestions about possible alternatives and a challenge to psychological researchers to develop new methods that actually assess the reliability of research findings.
>  
> And, to simply partition this problem off to psychology seems to be a significant mistake, as it appears medicine is afflicted just as badly with what looks to be the same approach to NHST.
>  
> Ioannidis, J.P. (2005). Why most published research findings are false. PLoS Medicine, 2, 8, 696-701.
>  
> Gelman, A., & Loken, E. (2014). The statistical crisis in science: Data-dependent analysis - a "garden of forking paths" - explains why many statistically significant comparisons don't hold up. American Scientist, 102, Nov, 460-465.
> It begins:
> “There is a growing realization that reported “statistically significant” claims in scientific publications are routinely mistaken. Researchers typically express the confidence in their data in terms of p-value: the probability that a perceived result is actually the result of random variation. The value of p (for “probability”) is a way of measuring the extent to which a data set provides evidence against a so-called null hypothesis. By convention, a p-value below 0.05 is considered a meaningful refutation of the null hypothesis; however, such conclusions are less solid than they appear.  ”
>  
> The articles I have quoted above are laden with evidence to support the various claims made by the authors.
>  
> Some of the facts and logic reasoning in these articles help explain why the BASP editors chose to ban NHST altogether.
>  
> Regards .. Paul
>  
> Chief Research Scientist
> Cognadev.com
> __________________________________________________________________________________
> W: www.cognadev.com
> W: www.pbarrett.net
> E: paul em pbarrett.net
> M: +64-(0)21-415625
>  
> From: medstats em googlegroups.com [mailto:medstats em googlegroups.com] 
> Sent: Monday, March 09, 2015 1:46 AM
> To: medstats em googlegroups.com
> Subject: Re: {MEDSTATS} Re: "Basic and Applied Psychology" Bans The Use of P Values and Confidence Intervals
>  
> One comment: 
>  
> You do not "accept your research hypothesis."
>  
> You do not reject it.
>  
> Martin
>  
> Martin P. Holt
>  
> Freelance Medical Statistician and Quality Expert
> 
> martinholt79 em yahoo.com
> 
> Persistence and Determination Alone are Omnipotent !
>  
> If you can't explain it simply, you don't understand it well enough.....Einstein
> 
> 
> 
> Linked In: https://www.linkedin.com/profile/edit?trk=nav_responsive_sub_nav_edit_profile
> 
> 
> From: Paul Barrett <paul em pbarrett.net>
> To: medstats em googlegroups.com 
> Sent: Saturday, 7 March 2015, 20:47
> Subject: RE: {MEDSTATS} Re: "Basic and Applied Psychology" Bans The Use of P Values and Confidence Intervals
>  
> I also want to say that what this NHST issue is really about, and what is at the heart of the BASP editorial:
>  
> From Gigerenzer, G. (2004). Mindless Statistics. The Journal of Socio-Economics, 33, , 587-606, p. 588
>  
> The null ritual:
> 1. Set up a statistical null hypothesis of “no mean difference” or “zero correlation.” Don’t specify the predictions of your research hypothesis or of any alternative substantive hypotheses.
> 2. Use 5% as a convention for rejecting the null. If significant, accept your research hypothesis. Report the result as p < 0.05, p < 0.01, or p < 0.001 (whichever comes next to the obtained p-value).
> 3. Always perform this procedure.
>  
> It’s not about how particular individuals might construe a null hypothesis, but about how an entire body of researchers ‘out there’ currently construe it in the context of that ‘null ritual’.
>  
> 
>  
> Regards .. Paul
>  
> Chief Research Scientist
> Cognadev.com
> __________________________________________________________________________________
> W: www.cognadev.com
> W: www.pbarrett.net
> E: paul em pbarrett.net
> M: +64-(0)21-415625
> -- 
> -- 
> To post a new thread to MedStats, send email to MedStats em googlegroups.com .
> MedStats' home page is http://groups.google.com/group/MedStats .
> Rules: http://groups.google.com/group/MedStats/web/medstats-rules
> 
> --- 
> You received this message because you are subscribed to the Google Groups "MedStats" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to medstats+unsubscribe em googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>  
> 
> -- 
> -- 
> To post a new thread to MedStats, send email to MedStats em googlegroups.com .
> MedStats' home page is http://groups.google.com/group/MedStats .
> Rules: http://groups.google.com/group/MedStats/web/medstats-rules
> 
> --- 
> You received this message because you are subscribed to the Google Groups "MedStats" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to medstats+unsubscribe em googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> -- 
> -- 
> To post a new thread to MedStats, send email to MedStats em googlegroups.com .
> MedStats' home page is http://groups.google.com/group/MedStats .
> Rules: http://groups.google.com/group/MedStats/web/medstats-rules
> 
> --- 
> You received this message because you are subscribed to the Google Groups "MedStats" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to medstats+unsubscribe em googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
-------------- Próxima Parte ----------
Um anexo em HTML foi limpo...
URL: <https://lists.ime.usp.br/archives/abe/attachments/20150308/bb3d4994/attachment.html>