Feeds:
Posts
Comments

Posts Tagged ‘publication bias’

Bring back the ‘philosophy’ in ‘Natural philosophy’ (Oct 2019)

Posted in Discussions, English, tagged , , , , , , , , , , , , , , , on 05/10/2019| 1 Comment »

Many of today’s scientists (incl. myself a lot of the time) have probably lost touch with some of the central tenets of being a scientist – instead titles, number of published papers and grant money brought in becoming more important than the societal impact of their publications and how much they contributed to human knowledge. A shoddy paper published in Nature/Science/Cell (especially if cited/talked about a lot) carries far more weight than a solid paper in a less glamorous journal. An academic who brings in grant money – doesn’t matter if he/she wastes it on shoddy or average research – is far more important (i.e. they will be promoted and bring in further funding easier as they already brought in some before) than one who chooses to concentrate on producing solid research but struggles to bring in money e.g. due to a lack of funding in their specific field or publishing papers in non-glamorous journals due to ‘non-exciting’ results as they didn’t add a spin to their conclusions (click here for other examples). Some of the papers published in prestigious journals in my field would not have been accepted if the senior authors of the same papers were the reviewers – many seem to apply a less stringent criteria to their own papers. The relationship between editors and some senior scientists is also opaque which is ultimately damaging to science. Image source: naturalphilosophy.org
Hell for academics and researchers (NB: The list is loosely ordered and is not an exhaustive one). Of course, inspired by Dante’s Nine levels/layers/circles of Hell

A few months ago, I spent almost a week trying to replicate a published “causal” association which had received >500 citations in the last 5 years. My aim was to provide a better effect estimate and to do this, I used two different datasets, one with similar and another with a larger sample size. However, both of my analyses returned null results (i.e. no effect of exposure on outcome). Positive controls were carried out to make sure the analysis pipeline was working correctly. Ultimately, I moved on to other ‘more interesting’ projects as there was no point spending time writing a paper that was probably going to end up in a ‘not-so-prestigious’ journal and never going to get >500 citations or be weighted heavily when I apply for grants/fellowships.

Consequently, inadvertently I contributed to publication bias on this issue – and no other analyses on the subject matter were published since the original publication, so I am sure others have found similar results and chose not to publish.

State of academia (very generally speaking): Really talented and successful people working like slaves for unimportant academic titles and average salaries. What’s worse is that the job market is so fierce that most are perfectly happy(!) to just get on with their ‘jobs‘ and do what they’ve always been doing (Note: this is my first attempt at drawing using Paint 🙂 )

However, I have changed my mind about publishing null/negative results after encountering Russell, Wittgenstein and others’ long debates on proving ‘negative’ truths/facts (and in a nutshell, how hard it is to prove negatives – which should make it especially important to publish conclusive null findings). These giants of philosophy thought it was an important issue and spent years structuring their ideas but here I am, not seeing my conclusive null results worthy of publication. I (and the others who found similar results) should have at least published a preprint to right a wrong – and this sentiment doesn’t just apply to the scientific literature. I also think academics should spend some time on social media to issue corrections to common misconceptions in the general public.

This also got me thinking about my university education: I was not taught any philosophy other than bioethics during my undergraduate course in biological sciences (specialising in Genetics in the final year). I am now more convinced than ever that ‘relevant’ philosophy (e.g. importance of publishing all results, taking a step back and revisiting what ‘knowledge’ is and how to attain ‘truth’, how to construct an argument1, critical thinking/logical fallacies, what is an academic’s intellectual responsibility?) should be embedded and mandatory in all ‘natural science’ courses. This way, I believe future scientists and journal editors would appreciate the importance of publishing negative/null results more and allow well-done experiments to be published in ‘prestigious’ journals more. This way, hopefully, less published research findings are going to be false2.

References/Further reading:

  1. Think Again I: How to Understand Arguments (Coursera MOOC)
  2. Ioannidis JPA. Why Most Published Research Findings Are False. PLoS Med. 2(8): e124 (2015)
  3. How Life Sciences Actually Work: Findings of a Year-Long Investigation (Blog post)
  4. An interesting Quora discussion: Why do some intelligent people lose all interest in academia?
  5. Calculating the ‘worth’ of an academic (Blog post)
A gross generalisation but unfortunately there is some truth behind this table – and it’s not even a comprehensive list (e.g. gatekeepers, cherry picking of results). Incentives need to change asap – and more idealists are needed in academic circles!

*the title comes from the fact that today’s natural scientists would have been called ‘natural philosophers’ back in the day

Read Full Post »