[ABE-L] Fwd: {MEDSTATS} Re: "Basic and Applied Psychology" Bans The Use of P Values and Confidence Intervals

[Basilio De Braganca Pereira]

Alexandra in action


Enviado do meu iPhone

Início da mensagem encaminhada

> De: "Sean O'Riordain" <seanpor em acm.org>
> Data: 12 de março de 2015 03:37:43 BRT
> Para: medstats em googlegroups.com
> Assunto: Re: {MEDSTATS} Re: "Basic and Applied Psychology" Bans The Use of P Values and Confidence Intervals
> Responder A: medstats em googlegroups.com
> 
> Xi'an (aka Prof. Christian P. Robert) has just blogged about this, and picked up on the fact that it is "inspired by a nihilistic anti-statistical stance"...
> https://xianblog.wordpress.com/2015/03/12/eliminating-an-important-obstacle-to-creative-thinking-statistics/
> 
> 
>> On 9 March 2015 at 13:30, Marc Schwartz <marc_schwartz em me.com> wrote:
>> 
>> > On Mar 9, 2015, at 6:35 AM, Paul Barrett <paul em pbarrett.net> wrote:
>> >
>> > I take it no-one here has read any of John Ioannidis’ work?
>> >
>> > For example, take a quote from:
>> > Ioannidis, J.P. (2010). Lies, damned lies, and medical science.. The Atlantic (http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/), November, 11, 1-10.
>> > “Ioannidis is what’s known as a meta-researcher, and he’s become one of the world’s foremost experts on the credibility of medical research. He and his team have shown, again and again, and in many different ways, that much of what biomedical researchers conclude in published studies—conclusions that doctors keep in mind when they prescribe antibiotics or blood-pressure medication, or when they advise us to consume more fiber or less meat, or when they recommend surgery for heart disease or back pain—is misleading, exaggerated, and often flat-out wrong.
>> >
>> > He charges that as much as 90 percent of the published medical information that doctors rely on is flawed.
>> >
>> > His work has been widely accepted by the medical community; it has been published in the field’s top journals, where it is heavily cited; and he is a big draw at pervasively flawed, and so riddled with conflicts of interest, that it might be chronically resistant to change—or even to publicly admitting that there’s a problem.”
>> >
>> > Perhaps a perusal of:
>> > Ioannidis, J.P.A., & Panagiotou, O.A. (2011). Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent meta-analyses. Journal of the American Medical Association, 305, 21, 2200-2210.
>> > might also suggest that how medical investigators are being taught by their professional statisticians is not necessarily having the desired effect?
>> >
>> > Who teaches neuroscientists their ‘statistical inference’ by the way?
>> > Button, K.S., Ioannidis, J.P.A., Mokrysz, C., Nosek, B.A., Flint, J., Robinson, E.S.J., & Munafò, M.R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews: Neuroscience, 14, 5, 365-376.
>> > It begins: “ It has been claimed and demonstrated that many (and possibly most) of the conclusions drawn from biomedi­cal research are probably false”
>> >
>> > Whether or not professional statisticians or others teach NHST to medical or social science students/researchers, the end result looks depressingly similar. Might it be that NHST itself is the problem, as suggested by others - those referenced in my previous email?
>> 
>> 
>> The problem is not with the NHST methodology itself, but the frequent lack of the implementation of any rigorous, prospective, experimental design methodology, likely due to the “publish or die” pressures that have coopted academic and other research, notably to produce research with "significant" positive findings.
>> 
>> This is one of the reasons that clinical trials now need to be pre-registered on ClinicalTrials.gov or similar sites, in order to inhibit the burying of studies with negative findings. Any reputable journal will now refuse to publish a study unless it was pre-registered, before any subjects are enrolled and the sponsors/PIs have obligations to publish at least some primary results on the registration site.
>> 
>> The last reference you point to regarding underpowered studies and small sample sizes, which I am aware of and have used myself, suggests:
>> 
>> From: http://www.nature.com/nrn/journal/v14/n5/full/nrn3475.html
>> 
>> "Research that produces novel results, statistically significant results (that is, typically p < 0.05) and seemingly 'clean' results is more likely to be published. As a consequence, researchers have strong incentives to engage in research practices that make their findings publishable quickly, even if those practices reduce the likelihood that the findings reflect a true (that is, non-null) effect."
>> 
>> 
>> The authors of that paper, propose various approaches to reducing the problem:
>> 
>>   http://www.nature.com/nrn/journal/v14/n5/box/nrn3475_BX2.html
>> 
>>   1. Perform an a priori power calculation
>>   2. Disclose methods and findings transparently
>>   3. Pre-register your study protocol and analysis plan
>>   4. Make study materials and data available
>>   5. Work collaboratively to increase power and replicate findings
>> 
>> not one of which, BTW, is to discard NHST.
>> 
>> As they reference in that paper, low powered studies are a waste of resources, not only financial, but of subjects as well, arguably putting subjects at risk, when there will be no scientific value to the research. Underpowered studies also tend to inflate effect sizes.
>> 
>> So, are you going to start restricting budgets and grants for improperly designed studies. Will you require more rigorous reviews, including statisticians, so that there is more money for properly designed research in order to cover the costs associated with enrolling more subjects?
>> 
>> Are you going to require competent statisticians be a co-author on a study?
>> 
>> Are you going to require that a competent statistician is part of any peer review process for a journal submission?
>> 
>> This is what makes the BASP editorial and the solutions that they propose absurd.
>> 
>> As I noted in a prior reply to this thread, simply publishing descriptive statistics is not the solution. You are not going to prevent skilled readers from calculating confidence intervals and inferring p values from that data.
>> 
>> They also proposed larger sample sizes, which is fine, but how do you get there (how much larger?) and how do you fund studies that are going to be more expensive as a result?
>> 
>> I have heard criticisms of NHST, but I have yet to see a proposal for a competent alternative.
>> 
>> Yes, there are machine learning methods, that have largely evolved out of computer science and not from classical statistics, that do not rely upon NHST, but on the minimization of some measurement of error. However, they tend to require much larger sample sizes. They are fine for applications such as "big data", where NHST is indeed not appropriate. But, they have their own limitations.
>> 
>> So, what is the solution Paul?
>> 
>> Regards,
>> 
>> Marc
>> 
>> 
>> >
>> > Regards .. Paul
>> >
>> > Chief Research Scientist
>> > Cognadev.com
>> > __________________________________________________________________________________
>> > W: www.cognadev.com
>> > W: www.pbarrett.net
>> > E: paul em pbarrett.net
>> > M: +64-(0)21-415625
>> >
>> > From: medstats em googlegroups.com [mailto:medstats em googlegroups.com] On Behalf Of Martin Bland
>> > Sent: Monday, March 09, 2015 11:30 PM
>> > To: medstats em googlegroups.com
>> > Subject: Re: {MEDSTATS} Re: "Basic and Applied Psychology" Bans The Use of P Values and Confidence Intervals
>> >
>> > One hypothesis worth testing is that it is taught by psychologists, not statisticians, as would be the case in medicine.
>> >
>> > Martin
>> >
>> > On 9 March 2015 at 10:19, John Whittington <John.W em mediscience.co.uk> wrote:
>> >> At 23:34 08/03/2015 -0400, Michael Cooney wrote:
>> >>> I think it's important to frame the scope of this debate.  These gross misinterpretations of statistics and poor planning of studies do not occur psychological studies makes me wonder if there isn't something inherently wrong with how we teach statistics to psychologists.
>> >>
>> >> I have to agree with all of that.
>> >>
>> >> Kind Regards,
>> >>
>> >>
>> >> John
>> >>
>> >> ----------------------------------------------------------------
>> >> Dr John Whittington,       Voice:    +44 (0) 1296 730225
>> >> Mediscience Services       Fax:      +44 (0) 1296 738893
>> >> Twyford Manor, Twyford,    E-mail:   John.W em mediscience.co.uk
>> >> Buckingham  MK18 4EL, UK
>> >> ----------------------------------------------------------------
>> >>
>> >> --
>> >> --
>> >> To post a new thread to MedStats, send email to MedStats em googlegroups.com .
>> >> MedStats' home page is http://groups.google.com/group/MedStats .
>> >> Rules: http://groups.google.com/group/MedStats/web/medstats-rules
>> >>
>> >> --- You received this message because you are subscribed to the Google Groups "MedStats" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send an email to medstats+unsubscribe em googlegroups.com.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> >
>> > --
>> > ***************************************************
>> > J. Martin Bland
>> > Prof. of Health Statistics
>> > Dept. of Health Sciences
>> > Seebohm Rowntree Building
>> > University of York
>> > Heslington
>> > York YO10 5DD
>> >
>> > Email: martin.bland em york.ac.uk
>> > Phone: 01904 321334     Fax: 01904 321382
>> > Web site: http://martinbland.co.uk/
>> >
>> > Statement by the University of York:
>> > This email and its attachments may be confidential and are intended solely for the use of the intended recipient. If you are not the intended recipient of this email and its attachments, you must take no action based upon them, nor must you copy or show them to anyone. Please contact the sender if you believe you have received this email in error. Any views or opinions expressed are solely those of the author and do not necessarily represent those of The University of York.
>> > ***************************************************
>> > --
>> > --
>> > To post a new thread to MedStats, send email to MedStats em googlegroups.com .
>> > MedStats' home page is http://groups.google.com/group/MedStats .
>> > Rules: http://groups.google.com/group/MedStats/web/medstats-rules
>> >
>> > ---
>> > You received this message because you are subscribed to the Google Groups "MedStats" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an email to medstats+unsubscribe em googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>> >
>> > --
>> > --
>> > To post a new thread to MedStats, send email to MedStats em googlegroups.com .
>> > MedStats' home page is http://groups.google.com/group/MedStats .
>> > Rules: http://groups.google.com/group/MedStats/web/medstats-rules
>> >
>> > ---
>> > You received this message because you are subscribed to the Google Groups "MedStats" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an email to medstats+unsubscribe em googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>> 
>> --
>> --
>> To post a new thread to MedStats, send email to MedStats em googlegroups.com .
>> MedStats' home page is http://groups.google.com/group/MedStats .
>> Rules: http://groups.google.com/group/MedStats/web/medstats-rules
>> 
>> ---
>> You received this message because you are subscribed to the Google Groups "MedStats" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to medstats+unsubscribe em googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> -- 
> To post a new thread to MedStats, send email to MedStats em googlegroups.com .
> MedStats' home page is http://groups.google.com/group/MedStats .
> Rules: http://groups.google.com/group/MedStats/web/medstats-rules
> 
> --- 
> You received this message because you are subscribed to the Google Groups "MedStats" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to medstats+unsubscribe em googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
-------------- Próxima Parte ----------
Um anexo em HTML foi limpo...
URL: <https://lists.ime.usp.br/archives/abe/attachments/20150312/d45dd3ee/attachment.html>