[ABE-L] Fwd: {MEDSTATS} Re: About professor Ioannidis and "Basic and Applied Psychology" Bans The Use of P Values and Confidence Intervals

[Basilio de Braganca Pereira]

Esse é o cara: Professor  Ioannidis


---------- Forwarded message ----------
From: Marc Schwartz <marc_schwartz em me.com>
Date: 2015-03-09 10:30 GMT-03:00
Subject: Re: {MEDSTATS} Re: "Basic and Applied Psychology" Bans The Use of
P Values and Confidence Intervals
To: MedStats MedStats <medstats em googlegroups.com>



> On Mar 9, 2015, at 6:35 AM, Paul Barrett <paul em pbarrett.net> wrote:
>
> I take it no-one here has read any of John Ioannidis’ work?
>
> For example, take a quote from:
> Ioannidis, J.P. (2010). Lies, damned lies, and medical science.. The
Atlantic (
http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/),
November, 11, 1-10.
> “Ioannidis is what’s known as a meta-researcher, and he’s become one of
the world’s foremost experts on the credibility of medical research. He and
his team have shown, again and again, and in many different ways, that much
of what biomedical researchers conclude in published studies—conclusions
that doctors keep in mind when they prescribe antibiotics or blood-pressure
medication, or when they advise us to consume more fiber or less meat, or
when they recommend surgery for heart disease or back pain—is misleading,
exaggerated, and often flat-out wrong.
>
> He charges that as much as 90 percent of the published medical
information that doctors rely on is flawed.
>
> His work has been widely accepted by the medical community; it has been
published in the field’s top journals, where it is heavily cited; and he is
a big draw at pervasively flawed, and so riddled with conflicts of
interest, that it might be chronically resistant to change—or even to
publicly admitting that there’s a problem.”
>
> Perhaps a perusal of:
> Ioannidis, J.P.A., & Panagiotou, O.A. (2011). Comparison of effect sizes
associated with biomarkers reported in highly cited individual articles and
in subsequent meta-analyses. Journal of the American Medical Association,
305, 21, 2200-2210.
> might also suggest that how medical investigators are being taught by
their professional statisticians is not necessarily having the desired
effect?
>
> Who teaches neuroscientists their ‘statistical inference’ by the way?
> Button, K.S., Ioannidis, J.P.A., Mokrysz, C., Nosek, B.A., Flint, J.,
Robinson, E.S.J., & Munafò, M.R. (2013). Power failure: why small sample
size undermines the reliability of neuroscience. Nature Reviews:
Neuroscience, 14, 5, 365-376.
> It begins: “ It has been claimed and demonstrated that many (and possibly
most) of the conclusions drawn from biomedi­cal research are probably false”
>
> Whether or not professional statisticians or others teach NHST to medical
or social science students/researchers, the end result looks depressingly
similar. Might it be that NHST itself is the problem, as suggested by
others - those referenced in my previous email?


The problem is not with the NHST methodology itself, but the frequent lack
of the implementation of any rigorous, prospective, experimental design
methodology, likely due to the “publish or die” pressures that have coopted
academic and other research, notably to produce research with "significant"
positive findings.

This is one of the reasons that clinical trials now need to be
pre-registered on ClinicalTrials.gov or similar sites, in order to inhibit
the burying of studies with negative findings. Any reputable journal will
now refuse to publish a study unless it was pre-registered, before any
subjects are enrolled and the sponsors/PIs have obligations to publish at
least some primary results on the registration site.

The last reference you point to regarding underpowered studies and small
sample sizes, which I am aware of and have used myself, suggests:

From: http://www.nature.com/nrn/journal/v14/n5/full/nrn3475.html

"Research that produces novel results, statistically significant results
(that is, typically p < 0.05) and seemingly 'clean' results is more likely
to be published. As a consequence, researchers have strong incentives to
engage in research practices that make their findings publishable quickly,
even if those practices reduce the likelihood that the findings reflect a
true (that is, non-null) effect."


The authors of that paper, propose various approaches to reducing the
problem:

  http://www.nature.com/nrn/journal/v14/n5/box/nrn3475_BX2.html

  1. Perform an a priori power calculation
  2. Disclose methods and findings transparently
  3. Pre-register your study protocol and analysis plan
  4. Make study materials and data available
  5. Work collaboratively to increase power and replicate findings

not one of which, BTW, is to discard NHST.

As they reference in that paper, low powered studies are a waste of
resources, not only financial, but of subjects as well, arguably putting
subjects at risk, when there will be no scientific value to the research.
Underpowered studies also tend to inflate effect sizes.

So, are you going to start restricting budgets and grants for improperly
designed studies. Will you require more rigorous reviews, including
statisticians, so that there is more money for properly designed research
in order to cover the costs associated with enrolling more subjects?

Are you going to require competent statisticians be a co-author on a study?

Are you going to require that a competent statistician is part of any peer
review process for a journal submission?

This is what makes the BASP editorial and the solutions that they propose
absurd.

As I noted in a prior reply to this thread, simply publishing descriptive
statistics is not the solution. You are not going to prevent skilled
readers from calculating confidence intervals and inferring p values from
that data.

They also proposed larger sample sizes, which is fine, but how do you get
there (how much larger?) and how do you fund studies that are going to be
more expensive as a result?

I have heard criticisms of NHST, but I have yet to see a proposal for a
competent alternative.

Yes, there are machine learning methods, that have largely evolved out of
computer science and not from classical statistics, that do not rely upon
NHST, but on the minimization of some measurement of error. However, they
tend to require much larger sample sizes. They are fine for applications
such as "big data", where NHST is indeed not appropriate. But, they have
their own limitations.

So, what is the solution Paul?

Regards,

Marc


>
> Regards .. Paul
>
> Chief Research Scientist
> Cognadev.com
>
__________________________________________________________________________________
> W: www.cognadev.com
> W: www.pbarrett.net
> E: paul em pbarrett.net
> M: +64-(0)21-415625
>
> From: medstats em googlegroups.com [mailto:medstats em googlegroups.com] On
Behalf Of Martin Bland
> Sent: Monday, March 09, 2015 11:30 PM
> To: medstats em googlegroups.com
> Subject: Re: {MEDSTATS} Re: "Basic and Applied Psychology" Bans The Use
of P Values and Confidence Intervals
>
> One hypothesis worth testing is that it is taught by psychologists, not
statisticians, as would be the case in medicine.
>
> Martin
>
> On 9 March 2015 at 10:19, John Whittington <John.W em mediscience.co.uk>
wrote:
>> At 23:34 08/03/2015 -0400, Michael Cooney wrote:
>>> I think it's important to frame the scope of this debate.  These gross
misinterpretations of statistics and poor planning of studies do not occur
psychological studies makes me wonder if there isn't something inherently
wrong with how we teach statistics to psychologists.
>>
>> I have to agree with all of that.
>>
>> Kind Regards,
>>
>>
>> John
>>
>> ----------------------------------------------------------------
>> Dr John Whittington,       Voice:    +44 (0) 1296 730225
>> Mediscience Services       Fax:      +44 (0) 1296 738893
>> Twyford Manor, Twyford,    E-mail:   John.W em mediscience.co.uk
>> Buckingham  MK18 4EL, UK
>> ----------------------------------------------------------------
>>
>> --
>> --
>> To post a new thread to MedStats, send email to MedStats em googlegroups.com
.
>> MedStats' home page is http://groups.google.com/group/MedStats .
>> Rules: http://groups.google.com/group/MedStats/web/medstats-rules
>>
>> --- You received this message because you are subscribed to the Google
Groups "MedStats" group.
>> To unsubscribe from this group and stop receiving emails from it, send
an email to medstats+unsubscribe em googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> ***************************************************
> J. Martin Bland
> Prof. of Health Statistics
> Dept. of Health Sciences
> Seebohm Rowntree Building
> University of York
> Heslington
> York YO10 5DD
>
> Email: martin.bland em york.ac.uk
> Phone: 01904 321334     Fax: 01904 321382
> Web site: http://martinbland.co.uk/
>
> Statement by the University of York:
> This email and its attachments may be confidential and are intended
solely for the use of the intended recipient. If you are not the intended
recipient of this email and its attachments, you must take no action based
upon them, nor must you copy or show them to anyone. Please contact the
sender if you believe you have received this email in error. Any views or
opinions expressed are solely those of the author and do not necessarily
represent those of The University of York.
> ***************************************************
> --
> --
> To post a new thread to MedStats, send email to MedStats em googlegroups.com
.
> MedStats' home page is http://groups.google.com/group/MedStats .
> Rules: http://groups.google.com/group/MedStats/web/medstats-rules
>
> ---
> You received this message because you are subscribed to the Google Groups
"MedStats" group.
> To unsubscribe from this group and stop receiving emails from it, send an
email to medstats+unsubscribe em googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
> --
> --
> To post a new thread to MedStats, send email to MedStats em googlegroups.com
.
> MedStats' home page is http://groups.google.com/group/MedStats .
> Rules: http://groups.google.com/group/MedStats/web/medstats-rules
>
> ---
> You received this message because you are subscribed to the Google Groups
"MedStats" group.
> To unsubscribe from this group and stop receiving emails from it, send an
email to medstats+unsubscribe em googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
--
To post a new thread to MedStats, send email to MedStats em googlegroups.com .
MedStats' home page is http://groups.google.com/group/MedStats .
Rules: http://groups.google.com/group/MedStats/web/medstats-rules

---
You received this message because you are subscribed to the Google Groups
"MedStats" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to medstats+unsubscribe em googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
-------------- Próxima Parte ----------
Um anexo em HTML foi limpo...
URL: <https://lists.ime.usp.br/archives/abe/attachments/20150309/7de1a0b7/attachment.html>