Friday, 26 August 2022

“Scientific proof” and “self-help”

The sun rose today, but will it rise tomorrow?
I mentioned in my last post that Peakrill Press is branching out into a variety of non-gaming related content. I will also be blogging more about random stuff. If you're here just for RPGs, that's cool: there'll still be plenty, and you can skip the rest. But I hope that, if you enjoy my writing about games, you'll appreciate my writing about everything else under the sun.

Below is an example of such other stuff. I'm thinking of writing - bear with me here - a self-help book. I already posted what I guess you could call some advice on productivity and, after 53 years struggling to stay cheerful and get things done, while also managing Bipolar Disorder, I feel I have useful advice to offer.

The passage below will be part of the book, either an introduction or an appendix. I'd especially appreciate any scientists giving it the once-over. As for scientismists, I'd love them to read it, though it may bring them out in a rash.
Any good scientist will tell you that there is no such thing as "scientific proof". What this inaccurate but handy phrase is shorthand for is “a scientific theory that has a lot of evidence that appears to back it up”. As the great philosopher David Hume teaches us, even the theory that “the sun rises every morning” is one we can never prove to be true, because: what about tomorrow morning?

There are folks, I’ll call them scientismists, some of whom believe that scientific proof is a thing, and almost all of whom believe that if a thing doesn’t have at least “a lot of evidence that appears to back it up” then it’s not worth doing, perhaps even dangerous. What a joyless approach to life!

What scientismists appear to forget (even though they often use the term!) is a thing called the “Placebo Effect” which basically means that even when a thing doesn’t work, it works! There’s a belief out there that if something - say, for example, homoeopathy - is “only” as effective as the placebo effect then it’s bunkum, to be avoided at all costs. What a wrong-headed approach to doing good!

Some of the stuff I’ll suggest here - in particular the habit of “gratitude journaling” that forms the heart of this book - have “a lot of evidence which appears to back them up”. Other stuff, like drawing trees or counting the number of petals on a daisy, probably don’t. That shouldn’t stop you from doing them. 
(Disclaimer: not everything untested is as good as a placebo. For example, the theory that jumping out of a plane without a parachute is better for you than jumping out with one has not yet been tested, and I don’t advise you to be the first to try).
By the way, there are some very good books out there which go into detail on which “self-help” practices appear to be more effective than the Placebo Effect -  I recommend starting with Richard Wiseman’s “59 Seconds”. But do bear in mind that all you need to do in order for something you do to do you good, is to believe that it does you good.

I say “all you need to do”. Forcing yourself to believe something is a tough nut to crack, especially if you’ve cynical tendencies like mine. But it is doable. Just remember: the Placebo Effect exists. And it’s magic!

8 comments:

  1. A few comments:

    There should be a distinction between things which have not been researched, vs. things which have been researched and are not found to be empirically valid. Empirical validity is not the end all be all regardless; you can have type-1 errors (finding a significant result in your sample even though the hypothesis is not "T"rue), or if there are sampling biases, or things change over time, etc., but it is worth being more skeptical about things which have been researched and not found to be empirically valid, than things which simply have not been researched.

    There is some truth in what you say about Placebo Effect, but it can be problematic if people are doing a thing that is not actually helpful above that baseline if there are better options, or doing that thing has other externalities. That said, on a personal level, if it feels like it's helping you and it's not hurting anyone else, then sure.

    Another thing worth thinking about, is that when we talk about empirical validation and scientific experimentation, often people will talk about the mean effect; drug A lowered anxiety by X% compared to comparison group (placebo) or control (no drug), but ignore the standard deviation, or other distributional attributes. Just because drug A on average lowered anxiety by X%, it may be that the outcomes were not normally distributed but bimodal; for some people it lowered anxiety and for others it actually increased anxiety, but on average it was lowered. There may be co-varying factors that weren't accounted for, like maybe the drug works differently for men vs. women (on average), and if you control for gender or take the partial effect, you find a significant effect only for one gender. Obviously, and without sidetracking too much, the fact that so much research around gender differences is still rooted in gender binary is also not ideal; I'm no longer involved in research but when I was last involved, that was slowly changing, and the slowness had more to do with trying to figure out the best ways to go about it, and also for reliability, than for any deliberate resistance, so hopefully that'll continue to be true.

    Getting back on track, the point in this is not to say that you should disregard the results of scientific research or as an excuse to lean in to your own confirmation bias, as much as it is to say, there are ways to engage with research that are more nuanced than just "whole-hearted Belief in 'Science'" vs. "uncritical rejection of science masquerading as skepticism". I don't mean to suggest you don't know that, just elaborating some of my own takes on what that means.

    ReplyDelete
    Replies
    1. Wow, that's *really* helpful, thanks Max. (Not so much helpful in the context of writing this, which as you've grokked is a deliberately dumbed down and somewhat straw-manny. But helpful in the context of, well, life and knowledge in general and "gee, that's interesting"). I appreciate the time it must have taken you to think & write it out.

      And, yeah, doing something untested of course doesn't mean it'll always be placebo-good or better, as made clear by perhaps the most famous "study" in this field - https://www.bmj.com/content/363/bmj.k5094 - I should probably add a footnote to this effect.

      To give slightly more context, around 10 years ago I would place myself squarely within the stereotype of "scientismists" that I describe. Patron saints: B Goldacre & R Dawkins ("Saints??? How dare you use that superstition-tainted word"). My views have shifted so far that they're almost but not quite the opposite (but informed by my adventures-in-science), so I'm trolling my old self.

      Even until recently though, I probably would have subscribed to the scientismists view on homeopathy (uncomfortably, in view of the number of friends I have who swear by it). But something I read recently made me espouse the viewpoint in this piece above; homeopathy harms nobody, and is actually, via placebo, beneficial for many! (I mean, it undoubtedly causes what could be classed as *some* financial harm, but that can be balanced against the harm caused by talented researchers wasting their lives arguing with homeopathy fans)

      Delete
    2. Ugh, I just noticed that Blogger justifies text in comments by default. Daaaaamn, that's ugly... though no way am I going to dive into this site's CSS to fix it.

      Delete
    3. FYI I updated the post to add a reference to parachutes 🙃

      Delete
    4. Obviously not everybody can be domain experts on every subject, nor be expert researchers, analysts, statisticians, etc.; but that said, basic research and data literacy, a basic understanding of what it means for something to be statistically significant, etc., is just really powerful.

      Like ya you can't be a domain expert in all fields, but if you have these basic literacies, you can read research in any field and should be able to at least somewhat parse it out- ideally it shouldn't have to be a matter of faith or belief or evangelism at all. If anyone is confronting science that way, they are making a fundamental error- science isn't the result, it's the method. This is one of many reasons why I hate the false binaries people present around science vs. X, especially because usually the people perpetuating those false binaries do not themselves have those literacies.

      Obviously there are other issues- having the time to read the research, having access to the research (much of which is grossly paywalled by so-called academic journals despite being in large part publicly funded), having time and access to learn these literacies, etc., and I understand not everyone can do that, but if you can't do that, at least know what you don't know, ya know?

      But anyway ya, regardless, this is interesting what you're doing and I'm excited to see how it progresses.

      Delete
    5. Oh yeah, you're not wrong there! although I think that those "false binaries", at least in the works of people actually worth reading and thinking about both from the science and the "woo" side of the "divide". For example I've been reading a fair bit of woo lately and, almost without exception, it is written by people who have at least a working understanding of what the scientific method is and where it is invaluable, but they're writing about experiences which are outwith the imagination of scientismists. Of course there are hard-woo folks as well as hard-scientismists (I'm getting to enjoy writing that word 😅), but although/because they're probably more prevalent among the general public than among folks contributing to the expanding human knowledge, they're both extremes that are rarely worth paying much attention to except inasmuch as to say, in a Graham Chapman voice, "Stop this. It's silly!"

      Delete
    6. Oops, should read "...I thank that those "false binaries" are less important than their apparent cultural weight might imply..." Or something like that.

      Delete
    7. Sheesh. Typo-heaven. I *THINK*. Sorry, I am sleep deprived and descending from a mania.

      Delete