17 U.S. Code § 107 - Limitations on exclusive rights: Fair use
Notwithstanding the provisions of sections 106 and 106A, the fair use of a copyrighted work, including such use by reproduction in copies or phonorecords or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright. In determining whether the use made of a work in any particular case is a fair use the factors to be considered shall include—
(1) the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes [all mirrored content falls under this clause, any ads present are mirrored from the original site, mirrored content earns me no revenue whatsoever];
(2) the nature of the copyrighted work [this is a unique resource for the mentally ill, and preservation of it can be argued to be incredibly important];
(3) the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and
(4) the effect of the use upon the potential market for or value of the copyrighted work. [absolutely none whatsoever, as the original work no longer exists anywhere else (outside of whatever bits and pieces archive.org managed to capture) - the original site was intermittantly completely unavailable for a extended period of time before its SSL certificate expired (and was never renewed), and eventually the site went offline for good, then finally the DNS records were removed at some point prior to May 2018, so at this point it is well beyond the 'dead and rotting' stage]
The fact that a work is unpublished shall not itself bar a finding of fair use if such finding is made upon consideration of all the above factors.

This domain is not controlled by Jerod Poore, and I will NOT continue redirecting traffic from this domain to crazymeds.us [as I formerly did] while Jerod continues with his immature temper tantrum over adblock or continues to fail to maintain his site, fucking over his entire community and countless visitors in the process. [belated clarification: with specific regards to the adblock drama I was referring to Poore at one point replacing his entire site with a single page complaining about the amount of revenue lost to users with ad blocking active, which is something that I took extreme exception to because this affected ALL visitors to the site regardless of if or if not they were actually using ad blocking]
This mirror is unfortunately incomplete (and very slightly outdated), as /CrazyTalk/ was not included when I scraped the site (it was far too large to scrape given the site's extremely poor performance, my wish to avoid worsening the poor performance further, and other factors). If you're looking for a replacement forum, I suggest visiting https://www.crazyboards.org/forums/. There are issues with many of the mirrored pages, I am working on identifying and fixing them, but I do not have the time to address every single issue at this moment (although by now the majority of these issues have been resolved). Dynamic content is obviously completely broken (this is beyond my control), and the loss of /CrazyTalk/ is quite bad given how much good user-generated info was on there, but you have Jerod to "thank" for that. Maybe I'll bring it back online at some point, but it wouldn't be the same as before. For now, I suggest visiting CrazyBoards instead.
Note (Oct 9 2018): Infrequent additional updates regarding the status of this site will be posted on https://info.crazymeds.net

how to determine if cited sources are any good

Crazymeds’ Guide to Psychiatric Evaluations | Common Crazy Med Crap Index | Terms, Abbreviations, Acronyms & Initialisms
Why do they think they know so much? | About Crazymeds | Where does their money come from?

1.  Introduction

Even though we cite a lot of sources in our articles, how do you know those sources are any good? And even if you trust us implicitly1, how do you know if other sites are using quality material? If a bunch of websites have a big citing-jerk where they all depend upon each other to validate their material2, then it’s probably all bullshit. But when you see links to articles in PubMed, then it has to be the good stuff, right?

Not necessarily. While all of The Literature that shows up in PubMed and similar repositories is technically peer-reviewed, I find it difficult to take certain publications and their peers seriously. Medical Hypotheses is on the level of the National Enquirer. They’re onto something 5% of time, if that. But when they are right, sometimes it can be jaw-droppingly amazing. While Complementary Therapies in Medicine and The Journal of Alternative and Complementary Medicine have plenty of decent articles about things like supplements and mindfulness, they also seem to publish practically anything that ‘proves’ herbs are good and drugs are bad3, ‘mmmkay?

Then there’s the familiar scenario where some editor or publisher sees a headline, tells a reporter about it, who then picks a copy / logs onto the website of JAMA or Science and skims a study involving a dozen mice, chocolate, and their immortal babies with telekinesis. Or something else that never pans out because the study was flawed, mice aren’t human, flukes happen, or the reporter has no idea what they’re writing about.

So, how do you go about determining if the research is any good? Or if the website / newspaper / magazine interpreted the paper correctly? What if you don’t understand the science? What if you’re pretty sure the journalist / website publisher / random Dick from the Internet has no clue about the science the paper is about? Can you determine if the paper upon which the article/assertion is based is any good?

Yes, you can.
If you’re short on patience attention time, there’s a no frills summary, put together by someone else, at the Crazymeds Tumblr.

2.  Rating the Research

2.1  Building a Strong Foundation

First you need to learn how to read papers. Reading The Literature - the term of art for papers, articles, books, studies, etc. - is an art in itself.

  • If your experience with The Literature is second- or third-hand via articles in publications or at websites that aren’t all that science-oriented (even if they have a Science section) and don’t know where to start, Angelo State University’s pamphlet How to recognize peer-reviewed (refereed) journals is a a good, if extremely basic guide.
    • Just pretend you’re writing a paper for college while reading it.
  • Next we need to look at what’s in specific parts of an article you’ll find at PubMed and elsewhere. If you’re not familiar with the basic structure of research papers, they have the following components in the following order:
    • Abstract - Summary of the article and each of its components.
      • The Abstract is often the only part of a paper you’ll find cited. I’m as guilty of that as most places, but I’m trying to do it less often.
        • Although I do have a bunch of dead-tree version of journals4, and I occasionally get access to the full text from people in universities, so I’ve often read entire articles that aren’t available to most people who don’t want to pay $15 (or more) to read a single paper.
    • Introduction/Background - The hypothesis; why the study is being done.
    • Methods/Methodology - How they did it.
    • Results - How it turned out.
    • Discussion / Conclusion - Translating the results to English and/or spinning them to match the hypothesis. See below.
    • Supplementary Data, Supplementary Material - Information overload.
      • If you thought all the tables, charts and graphs were too much, don’t look here.
      • If you thought they weren’t enough, this is what you want.
    • Funding, Acknowledgements - Who paid for it. You won’t see this on many most older papers.
  • If, or once you have some idea of what The Literature is all about, you must read Trisha Greenhalgh’s5 series of excellent articles for the BMJ about how to find, read, and interpret research papers.
  • Most papers, especially those written since 2008 (or so) are written rather clearly. Anything using really obtuse terminology and obfuscating methodology is probably trying to cover up flaws and failures.
    • This is one way people attempt to make the results seem more favorable to the hypothesis, spinning them, than they actually were.
      • Without altering the data, which is a huge no-no.
    • Some people just can’t grasp the concept that failure is an option in science.
    • Usually it’s those who are taking drug company money.
  • Of course there will be papers using techniques and/or technologies that only specialists are going to understand.
    • If you don’t know the difference between HPLC and HAM-D, at least I can help you with HAM-D.
    • If you think you can teach yourself about technical stuff you don’t now know, go for it! I have no idea if it’s worth it, you’re the only one who can make that decision.

2.2  Do the Math

Most studies and all trials involve a lot of statistics. Here are a couple of articles to either refresh your memory or teach you how to interpret the statistics commonly used in all medical studies:

Once you have an idea of how to read papers, you can read some of the ones I have referenced all around Crazymeds - I try to link to the full-text as often as possible - and see if they make any sense. Better yet, see if I interpreted them correctly and let me know if you think I’m wrong and why. Better yet, if you have an account on the forum, open a topic here. I’m usually pretty good about giving credit to anyone who corrects my dumbass mistakes. And since I keep the full history of edits for every page, it’s really easy to see what changes have been made and when6.

Now we can get into determining if an article or webpage citing a paper, or the paper itself, is any good. We’ll start with the easy stuff.

2.3  Age Before Beauty

This is really easy way to see if a cite is any good.

  • How old is it? Age won’t necessarily invalidate a paper. We know that lithium treats bipolar disorder.
    • While we also know a hell of a lot more about using lithium, we still don’t know how the hell it works when it comes to bipolar disorder.
    • There’s no hard and fast cut-off date for a paper.
      • When writing about something that is still be researched, such as how lithium works, I try not to use anything older than five or six years for a single reference in a small piece, and eight-to-ten years when there will be several cites in a more detailed piece.
      • Unless there is a reason for some historical data. Like to illustrate how pure racism prevented Serpasil (reserpine) from reaching the market, and schizophrenics, for over 10 years.
    • But if the only paper cited on a webpage is over 30 years old, take a look at PubMed yourself.
      • You’re probably not going to find much, and what you do find will likely be more recent and less favorable than the one study referenced.
  • Rule of thumb: if an article, webpage, etc. cites only one or two papers that are over 20 years old, the odds are whatever they’re selling is a load of crap.
  • You can find the publication date of an article immediately above its title.

2.4  Size Matters

Bigger is better. It really is that simple.

  • This doesn’t mean that small studies are invalid or without credit.
    • Because there isn’t any point of testing a new hypothesis with an expensive study involving huge number of participants.
    • But you don’t want to rewrite your treatment plan, let alone the rules of anything, based on a study of 20 (or fewer!) people7.
  • The best studies have multiple groups of participants. For studies involving meds these are usually:
    • The control group, those who get a placebo (AKA a sugar pill).
      • Or, better yet, an active placebo, such as an older med already on the market.
    • And the experimental group, those who get the drug being tested.
    • Many of the studies on this site test multiple meds (AKA cage matches). These may or may not include a placebo.
      • When you have five or more meds involved, a placebo is kinda moot.
    • When comparing things you need at least 20 people in each group to get anything close to statistically significant data.
  • One really important thing: when you have experimental and control groups, they need to be roughly the same size!
  • A recent experiment of media hacking proved how badly I needed to point out the size issue.
  • The number of participants is often in the Abstract.
    • Sometimes the number in each group is given in the Abstract as well.
  • You can always find the number of participants in the Methodology section.
    • If it’s not in either, the study is probably almost certainly garbage.

2.5  Money Changes Everything

Who paid for the study can make a huge difference in the results. Unfortunately. And Science backs that up that depressing assertion to the tune of it being 3.6 times more likely for the results to favor the drug of whoever sponsored the study8.

  • If you skip down to the end of the full text of an article you’ll find the Funding and/or Acknowledgement sections.
    • Either or both of those will tell you who paid for, or contributed in some way to, the research.
    • If a drug company did more than supply meds and not-particularly-expensive equipment, and one of its meds is the subject of the research, there’s going to be some bias in favor of its product.
    • Which is another reason why I like to have new meds tested against existing ones.
      • If you have two (or more) drug companies paying for a clinical trial, there goes the bias in favor of any one med!
  • Any drug company-funded study is likely to be biased.
    • This doesn’t necessarily invalidate all of the data, but you need to take the bias into consideration when judging if the study is going to be any good for your purposes.
  • Author information now has their personal conflicts of interest (COI), which includes things like any bribes honoraria they’ve received from drug companies, any stock they own in which companies, if they’ve worked as consultants for them, and so forth.
    • From the studies I’ve read, which is a small percentage of those written about a subset of therapies to treat conditions that afflict, at most, 20% of people, personal COI doesn’t seem to cause that much of a bias.
    • The one exception being the most obvious: employees. If there’s an employee of a drug company on the team, you can bet your ass the results will be favorable.
    • Where author COI is depends on the age (it’s not in older papers) and format.
      • It’s only in the full text.
      • Online in HTML you’ll click on something somewhere, as there is no standard for it.
      • In .pdf files it’s usually at the bottom of either the first or last page.

2.6  Relevant Determinants

Now let’s get to the less easy stuff, using each component of a paper to determine if the cite is relevant or the paper is any good.

  • Before you even read the paper, the first thing to look at is who published it.
    • As mentioned above, some journals, such as Medical Hypothesis, are questionable at best.9
    • Turns out there are a couple of lists of journals and publishers that will publish anything10.
    • If the subject of the paper has little, if anything, to do with what the journal is about, you have to wonder why the author(s) submitted it.
    • Or if the publication isn’t in the country, let alone language, of the people who wrote the paper.
      • Unless, of course, there aren’t any journals on the subject published where the researchers are based.
      • Or the paper was submitted to a journal published in a predominately English-speaking country.
        • Especially if the researchers are based in a country/countries with close ties to the country where the journal is (Commonwealth members, e.g.).
      • Because English, not Latin, is currently the true language of Science.
      • And most of the world’s largest pharmaceutical companies are based in either the US or UK. And if you’re doing medical research that makes a big difference.
      • Basically, this rule doesn’t apply for anything published in English-language journals.
  • Next is where the researchers work. You’re looking for anyone who works for a drug company.
    • As mentioned above, if anyone on the research team works for a drug company, the outcome is practically guaranteed to be in favor of the drug company’s product.
    • However! There may still be plenty of useful data there, such as pharmacokinetics.
  • What type of paper it is, or what it is about can determine if having it written by someone who works for a drug company is good, bad, or neutral.
    • If it’s comparing how well their med works with one or more meds someone else made, the odds are their drug will win. Surprise!
    • If the paper is about a drug’s pharmacology, i.e. how it works, then you might have something that’s excellent or it’s a piece of shit, so it doesn’t matter who paid for it.
      • The date is often a dead giveaway, with more recent papers being far better.
      • As is the title. If it has “Phase I Clinical Trial” in the title, the odds are it’s worthless.
      • But check the Methodology anyway, because it might be a decent trial.
  • Abstract - The paper cited doesn’t have anything to do with the webpage that linked to it.
    • This covers both the first and fourth of the four basic questions of Critical Appraisal of Research in Evidence-Based Medicine:
      • “1. Does this study address a clearly focused question?”
      • “4. Are these valid, important results applicable to my patient or population?”
    • The authors of the paper could have won a Nobel Prize for their work in quantum mechanics, but that doesn’t matter if it was cited by a webpage about homeopathy.
    • I see this sort of thing all the time. Frequently it’s an error that’s funny for its WTFery, often it’s not an unintentional cite/link.
    • If the abstract doesn’t make any sense, the paper is useless as far as you’re concerned.
      • Again: is it a clearly-focused question?
      • If you know the subject and it’s gibberish, then the research is probably bogus.
      • If you don’t know anything on the subject and it’s gibberish, then the only way for you to tell if the research is any good or not is to validate their stats and methodology.
  • Introduction / Background - As with the Abstract - does the paper have anything to do with what the page (or anything else on the site) that linked to it.
    • On older papers that don’t have Funding and/or Acknowledgement sections, and/or author COI information, who paid for the study, if the authors work or worked for drug companies, and other potential conflicts of interest might be here.
  • Plain Language Summary - Something you’ll find on some newer papers.
    • By newer I mean 2008 or later, but I haven’t seen it on many crazy meds-related papers published prior to 2011.
    • It’s usually included with the abstract, but this practice wasn’t common until 2011.
    • Plain Language Summary is the Abstract and Introduction / Background sections combined and translated into English.
      • Sometimes other languages as well.
    • If you’re really lucky the Discussion / Conclusion gets summarized in it as well.
  • If the study is related to the article, skip down to…
  • Discussion / Conclusion - When you read this, does it make any sense?
    • Just like the Abstract, anything you don’t understand is either useless to you and you need to trust the author of the webpage that linked to the paper in question, or the research is worthless.
    • I’ve read a few papers where the results used obtuse statistics and 90% of the discussion was about how the people given the med fit into the confusing stats and nothing about how it helped them, or not, in any way.
    • Anyone who tries to make their article more difficult to understand in the Discussion section instead of making it easier to understand is hiding something.
    • If the results make sense, are they relevant?
      • Success or failure of an experiment does not necessarily determine relevance.
    • In the words of Critical Appraisal of Research in Evidence-Based Medicine:
      • “3. Are the valid results of this study important?”
    • If the results aren’t relevant, the study is bullshit. Test over.
    • If they are, let’s see if they were reached in a valid manner. Go back up to…
  • Methods/Methodology - This is where you need to apply some critical thinking skills. Things to look at:
    • The type of article it is (see Taxonomy below).
    • The number of participants in the study/trial.
      • As Dr. Bohannon made abundantly clear in his article, not including how many people were involved in the methodology section is an obvious red flag of a bogus study12.
    • How long it lasted.
    • Med vs. placebo, other med(s), or both?
    • Inclusion & exclusion criteria - what qualified people for, or disqualified them from participating? These can make or break the clinical trials that lead to drug approvals.
    • Which assessment tests were used. If you look at a lot of the really complete PI sheets for anything approved as a mood stabilizer you’ll see references to MADRS and YMRS, tests so vague that everything from lithium orotate to color therapy shows positive results.
    • Statistical methodology used.
      • Assuming you remember enough from your high school math classes13.
      • And/or this paper makes sense to you.
      • And/or you at least read one of the papers on stats in the series on reading papers mentioned above.
      • If you can judge whether or not someone used a bogus statistical methodology, then you can call bullshit on all sorts of research without having to know all that much about what is being studied!
      • Even if you don’t know diddly about statistics there’s one easy way to tell: anything with fewer than 20 people taking a drug (or whatever therapy is being tested) is utterly worthless.
        • 20 is the bare minimum required to get any sort of statistically relevant results in anything.
      • Once again, from Critical Appraisal of Research in Evidence-Based Medicine:
      • “2. Did the study use valid methods to address this question?”
        • If it fails just one of the four questions, it’s not much use. Test over.
      • If it does have more than 20 people and you can’t do the most basic statistical analysis to save your life, you’ll have to decide to take it on faith or not.
        • You either trust the author of the page on that point or you don’t.
  • Results - I am so not qualified to call bullshit on anyone’s research when it comes to the actual outcome.
    • For the most part.
    • If something smells wrong, I don’t use it.
      • Unless the paper really stinks, which makes it funny. I’ll include really bad studies for humor value.
      • Or to point out the corruption of science by drug company money.
      • Or both.
    • There is one way to tell if something is a bit off without having to know Jack Squat14 about statistics, meds, or anything else: the extent of drug company involvement.
      • Drug companies will often supply the drugs directly to researchers. There is nothing wrong with that.
      • Drug companies pay for (sponsor) studies a lot of studies. When that happens the study is 3.6 times more likely to favor the drug made by whoever paid for the study.
        • As mentioned above, that doesn’t necessarily invalidate all of the data (pharmacokinetics data are difficult, but not impossible to spin), but you need to take into account a heavy bias in favor of their drug when evaluating the results.
        • What can be a bigger problem is they won’t allow some of the data to be published.
        • Incidental data that are often overkill in these studies is no big deal.
        • But when there are lots of holes filled in with notes about the data being stored on record in Eli Lilly’s vaults due to trade secrets, then you probably shouldn’t trust that study.

These will stick around longer than most side effects. More ways to be stuck-up at Straitjacket T-shirts. All stickers $5 each. Available in packs of 10 and 50.
Medicine Is The Best Medicine stickers at Straitjacket T-shirts
Medicine Is The Best Medicine
Vaccines Cause Immunity stickers at Straitjacket T-shirts
Vaccines Cause Immunity
Mental Illness is NOT Contagious stickers at Straitjacket T-shirts
Mental Illness is NOT Contagious
Medicated For Your Protection stickers at Straitjacket T-shirts
Medicated For Your Protection

2.7  Taxonomy

What type of article is it? Here are the typical kinds of articles you’ll find cited on Crazymeds and other sites, with a brief description. I’ve ordered them from least to most useful for my work on Crazymeds. The generally accepted hierarchy is part of the super-brief version of this article at the Crazymeds Tumblr.

  • Letter (AKA lazy case report) - A doctor writes to a journal to tell them about this patient they had that you would not believe.
  • Case Report - A well-documented account of a doctor’s patient(s) that was just un-fracking-believable.
    • If there’s more than one un-fracking-believable patient it’s technically a Case Series, but PubMed still calls it a Case Report.
    • Letters and Case Reports/Series fall under the category of Anecdotal Evidence. Many researchers consider anecdotal evidence to be worthless. I don’t.
      • Mainly because all the consumer experiences with meds that I glean from the Crazy Talk forum, assorted review sites and support groups, and other sources are also anecdotal evidence. Assuming lots of overlap, especially with negative experiences, I still end up with more people than all the clinical trials I can find for many meds.
    • Prof. Greenhalgh, who is also a strong proponent of evidence-based medicine, writes that anecdotal evidence has its place as research.
  • Meta-Analysis15 - Someone gets grant money to have an un-/under-paid grad student go through all of The Literature looking for studies, etc. to support the hypothesis in the grant proposal.
    • A Meta-Analysis is essentially a study of studies, trials, reviews, etc.
      • The main problem I have with meta-analyses is how they have taken over The Literature. It’s like half of what’s published today is in the form of a meta-analysis.
      • They’re turning peer-reviewed journals into Twitter and Tumblr - nothing but retweets and reblogs16.
      • Hello! Where’s the original research? If we all turn into aggregators, the feeds will soon be empty.
    • Also: It’s really easy to skew the stats in a meta-analysis so they come out to prove the hypothesis.
    • All the letters indicate how crazily popular meta-analyses were twenty years ago. It’s so much worse now.
  • Open-Label or Observational Study - Anywhere from 5 to 10, rarely more than 20, patients at a hospital, clinic, etc. are given a drug and everyone knows what the drug is and why people are taking it.
  • Review17 - A less-formal meta-analysis written by someone who needed to keep their name in print (publish or perish), fill space in their journal, accept drug company money, or all of the above.
  • Single-Blind Study - Typically the same size as open-label, only the people taking the pills don’t know if it’s the med being tested or a placebo.
  • Double-Blind Study - Neither the people administering the pills nor the people taking them know which is the real thing and which is the placebo. Most have over 20 participants, but you need at least 20 in each group to make it worth the effort.
    • An incredibly precise double-blind study is known as a “double-dummy” study. In double-dummy studies the pills are the same size, color, etc. and everyone gets the same number of pills, and takes the same doses at the same time.
      • That’s probably a given in most double-blind studies, but you need to be really careful to not mix up the placebo with whatever is being tested.
    • A med passing a double-blind study is the bare minimum for me to suggest talking to your doctor about an off-label prescription if the regular stuff isn’t working for you18.
    • Anything with only less than this (single-blind, open-label, case report) should be reserved for when you’re running out of options.
  • Randomized Control Trial (RCT) - The gold standard of drug tests.
    • RCTs are basically large Double-Blind Studies with greater adherence to protocol to make sure absolutely nobody knows who was assigned to the experimental group (the people getting the drug being tested) and who was assigned to the control group (the people getting the placebo).
    • More RCTs are now “three armed” i.e. the participants are randomized (randomly selected) to be in one of three groups: placebo group or either of two drug groups.
    • Why two drugs? The European Medicines Agency requires a drug to be at least as good as an “active placebo” - another med, usually an older one, used to treat whatever condition the people have.
      • What’s the point in approving new meds that work better than a placebo?
      • SGAs should work better than Thorazine at a minimum, right?
    • Why a placebo as well?
    • Guess what the FDA requires?
      • “Your new antidepressant is better than Prozac? That’s nice. How did it do when tested against a placebo?”
  • Clinical Trial - A super-sized RCT. Hundreds people in each group, all of whom had to meet very specific criteria to participate.
    • Clinical Trials are for one purpose: to bring a new med to market.
    • There are several types, or phases, of Clinical Trials:
    • Phase I: Safety, Pharmacokinetics19, Side Effects.
      • Phase I usually has around 20 people.
    • Phase II: Is it effective? Figuring out what the dosages and dosing schedule should be.
      • The efficacy and dosing trials can be broken up into IIa and IIb, but you frequently need to know how much to take, and when to take it, in order to determine how effective a drug is going to be.
    • Phase III: Is it effective enough to put up with whatever the side effects are? And make money?
      • Phase III is broken down into a and b. IIIa is the truly super-sized RCT. If these go well the drug company will submit the paperwork to the FDA to claim they have a new med to treat a real20 condition.
      • If the FDA is satisfied, phase IIIb testing begins. These tend to be longer, follow-up versions of the IIIa tests that are hybrids of RCTs and large, months-to-year-long open-label studies21.
      • Thanks to all of the class-action lawsuits over known side effects, Phase IIIb studies now go on forfuckingever. Expect them to go on for even longer.
    • Phase IV: After it’s already being sold, are there any problems for people who take it longer than the Phase IIIb trials lasted? Like a few years?
  • Multicenter Clinical Trial - The same clinical trial taking place in several locations at the same time, often in different countries.
  • Systemic Review - What a Meta-Analysis is supposed to be.
    • No not really.
    • In a Systemic Review someone goes through all the RCTs on a treatment to:
      • Double-check the methodologies and results
      • Compare them against each other
      • Determine if there is any bias
        • Drug company sponsorship of the study, current or former drug company employees participating in the study, that sort of thing.
      • And grades them in terms of being evidence-based.
  • Longitudinal Study or Cohort Study - After a lot of RCTs and clinical trials they track the people who were taking the med being studied for a year or so. That’s just part of a really good study or trial. A longitudinal study is when they track a group of people who were in one of those, or who just take a med, for several years. Or longer.
    • Longitudinal / Cohort studies are frequently observational and don’t deal with treatments.
      • If you’re familiar with the Up series of movies you know about one of the longest observational cohort studies on record.
    • Longitudinal Studies are the sort of things that determine which meds are either the first ones to try or the ones to talk to your doctor about once your symptoms are under control on a med you don’t particularly like.
      • Especially if you have a condition where taking meds is likely to be a long-term, if not lifelong situation, such as epilepsy or schizophrenia.
  • Turns out pseudoscience has a similar hierarchy of ‘evidence’. Good for them!

3.  Quest for Knowledge

Ready to do some research on your own? See our General Bibliography Page for tips on finding papers and repositories of free full-text and neuropsychopharmacology-specific research sites 22. Even though teh Interwebs are full of ‘peer-reviewed’ journals that aren’t, not everyone will have access to the good stuff, and will have to rely on what is available for free. So here are some sites with large repositories of free full-text articles from reputable journals that really are peer-reviewed:

  • PubMed Central is the National Institute of Health’s repository of full-text articles and even online books that are available free of charge. These are your tax dollars at work, people, make good use of them.
  • PLoS Medicine. The Public Library of Science is probably the best-known repository of free, peer-reviewed papers and articles.
  • Standford University’s Highwire over 2 million papers available.
  • The advent of Google Scholar has made finding papers both easier and vastly more frustrating.
    • Easier because it’s basically regular Google that has filtered out most of the crap that doesn’t meet peer-review specifications.
      • By “most” I mean “random websites that mention whatever you entered” and not “fraudulently peer-reviewed sites”.
    • More frustrating because it’s Google. Which means…
      • it includes all sorts of crap that shouldn’t be there;
      • still omits stuff you expect Google to find;
      • returns dead links;
        • repeatedly;
      • its advanced search, while much improved, still doesn’t come close to PubMed’s sophistication;
        • “sophistication” also means “difficulty of use for anyone who hasn’t spent years in the data mines”;
      • and indicates papers are free full-text when they aren’t at a ratio of at least 5:1 when compared to PubMed.
  • Cochrane Database of Systematic Reviews Not everything is free, but there is a lot of free material here.
    • What makes this a key site is that it’s a collection of RCTs articles go through a second review process to check for bias (drug company sponsorship, e.g.) and the degree of evidence-based research.
  • The Agency for Healthcare Research and Quality (AHRQ) Evidence-Based Reports. Similar to the Cochrane Database above, this is a collection of free eBooks published by the AHRQ, which is part of the US Department of Health and Human Services. Each book is a systemic review of hundreds of studies on a particular subject.

4.  Just Because It’s Peer-Reviewed Doesn’t Mean It’s Right

Hey, after all that, you’re telling me a peer-reviewed paper in Nature or The New England Journal of Medicine could still be full of shit?!?

  • Yes. Sorry. The world is an imperfect place.
  • Want to be even more depressed about it? Who’s Afraid of Peer Review? details the lack of scrutiny at many online ‘peer-reviewed’ journals.
    • Repeat to yourself “Just because it’s on the Internet doesn’t make it true” until it’s engraved on your DNA.
    • And that you read the above on the Internet23.
  • Peer-review is a major part of the checks-and-balances system that separates Science from partisan political disinformation faith-based ‘education’ the delusional fantasy world so many people live in, yet I’m the one who’s crazy?!?!? whatever the fuck Intelligent Design is supposed to be believing something is true because that’s what a “wise man” said a couple hundred years back so shut your mouth before I shut it for you.
  • And even when it works, well, these two PLoS Papers sent ripples up and down the irony-paradox spectrum:
  • Here’s a third: Scientific method: Statistical errors
    • Every clinical trial I’ve read uses the P value. Now I get to wonder if that number means anything.
  • File under Follow the Money or No Shit, Sherlock: Scope and Impact of Financial Conflicts of Interest in Biomedical Research.
    • The answer is 3.6
    • Pay for a study and it’s between three and a half and four times more likely that the results will be in your favor.
      • I need to find more studies to both validate that result (HAH!) and either confirm or get get another number to work toward a consensus.
    • It’s difficult to tell to what extent that is still the case.
      • On one hand, that study is old (2003), and was done before the greater transparency in funding and conflicts of interest were routinely published.
      • OTOH, Retraction Watch (see below) indicates that the numbr of fraudulent papers being published is increasing.
  • Ever wonder what the hell someone meant by some bullshit like, “It’s the notes they didn’t play.”? Well, unfortunately it’s often the studies and clinical trials that aren’t published that are really telling about a med.
    • I’m no longer surprised at the extent to which medical research has been perverted by profit.
  • If a paper is rejected by a journal, some researchers will go journal-shopping and submit their paper to one after another until it meets somebody’s ever-lower standards.
    • I.e. the authors will pay someone to get it published.

4.1  It Gets Worse

The publish or perish culture has created an environment where too many24 researchers will fudge the numbers to get published.

Stick to your treatment plan with buttons and magnets. 2.25″ $4 & 3.5″ $4.50 at Straitjacket T-shirts
Pile of Pills buttons at Straitjacket T-shirts
Pile of Pills
Vaccines Cause Immunity buttons at Straitjacket T-shirts
Vaccines Cause Immunity
Medicated For Your Protection magnets at Straitjacket T-shirts
Medicated For Your Protection
Fuck Bipolar buttons at Straitjacket T-shirts
Fuck Bipolar

5.  Check your Balance

So, if you can’t trust peer-review, what can you trust?

  • There is a third part of Science’s checks and balance system: when multiple researchers come up with different results, and everyone gets to debate who is right.
    • More importantly, more people get to apply for more grant money to figure out who’s right.
    • I call this process “dueling studies.”
    • If someone cheated, or the paper shouldn’t have been published in the first place, dueling studies will out the fakers.
      • Think of it as trial by combat for the modern age.
    • Which is why I do my best to find and cite legitimate contradictory data.
      • I.e. Nothing like the Earth is only 6,000 years old or similar crap regarding vaccines.
    • And when new data come in I notice new data and have my act together to update the appropriate pages, I do so.
    • For example on the Advantages of Using AEDs as Mood Stabilizers Page, I wrote how I’ve changed my mind about two things that were a huge part of my core belief in psychopharmacology for close to a decade, because of new data:
      • APs are neuroprotective against bipolar kindling.
      • APs aren’t nearly as dangerous to mix with alcohol as I thought. In fact, they are also neuroprotective against alcohol-induced brain damage.
      • Heavy drinking + SGAs could make your liver explode a lot sooner than it would have without the APs, but heavy drinking + unmedicated batshit crazy will probably kill you a lot sooner than that.
    • Sorry. Where was I? Was I trying to make a point?
  • So even in those extremely rare cases where a paper shouldn’t have passed peer review but did, Science will fix its mistake.
  • Also: The rise of Evidence-Based Medicine (EBM) and Evidence-Based Research, along with greater transparency regarding funding, is bringing back the integrity to peer-reviewed research.

6.  Don’t Bother Reading All About It

No matter what, newspaper and non-peer-reviewed magazine articles are not sources for backing up claims about treatment options. If it doesn’t come from the source of the research material it’s not a citable source25.

My son and I were discussing what qualifies as good research and how you can tell the difference when he brought up newspaper and magazine articles based on studies. It seems like most journalists, either thinking they understand more than they know about the subject, or stuck with the job of writing about something they know they don’t understand all that well, do no more than skim the Introduction and Conclusion sections of a study, and write an article that gets it half-right at best, and mostly wrong most of the time. The Chocolate Bar Weight Loss media hack exposed just how many so-called journalists really are that lazy. Stop tweeting about what you’re doing and pay attention to what you’re doing!

As for books, a major source of Crazymeds’ information, trusting the source comes down to the authors’ and publishers’ reputations, and the authors’ research, publishing, and actual medical practice (if any) histories. While the links to Amazon go through my store so I can make a couple of bucks26 now and then, the main reason for those links is so you can see if I’m using what you’d consider to be quality source material. Are Amazon reviews and author information the best source for that? Not always, but, like the peer review system, it’s something that everyone can agree on.

Keep Crazymeds on the air.
Donate some spare electronic currency
you have floating around The Cloud
Display your mad pride.    
Tell your imaginary friends about Crazymeds.

7.  References

  1. Trisha Greenhalgh, Rod Taylor “How to read a paper: Papers that go beyond numbers (qualitative research)” BMJ 1997;315:740 (Published 20 September 1997)
  2. Trisha Greenhalgh “How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses)” BMJ 1997;315:672 (Published 13 September 1997)
  3. Trisha Greenhalgh “How to read a paper: Papers that tell you what things cost (economic analyses)” BMJ 1997;315:596 (Published 6 September 1997)
  4. Trisha Greenhalgh “How to read a paper: Papers that report diagnostic or screening tests” BMJ 1997;315:540 (Published 30 August 1997)
  5. Trisha Greenhalgh “How to read a paper: Papers that report drug trials” BMJ 1997;315:480 (Published 23 August 1997)
  6. Trisha Greenhalgh “How to read a paper: Statistics for the non-statistician. II: “Significant” relations and their pitfalls” BMJ 1997;315:422 (Published 16 August 1997)
  7. Trisha Greenhalgh “How to read a paper: Statistics for the non-statistician. I: Different types of data need different statistical tests” BMJ 1997;315:364 (Published 9 August 1997)
  8. Trisha Greenhalgh “How to read a paper: Assessing the methodological quality of published papers” BMJ 1997;315:305 (Published 2 August 1997)
  9. Trisha Greenhalgh “How to read a paper: getting your bearings (deciding what the paper is about)” BMJ 1997;315:243 (Published 26 July 1997)
  10. Trisha Greenhalgh “How to read a paper: The Medline database” BMJ 1997;315:180 (Published 19 July 1997)
  11. William Harris “How Scientific Peer Review Works” science.howstuffworks.com. (Published online 05 January 2009)
  12. Neal S Young, John P. A. Ioannidis, and Omar Al-Ubaydli “Why Current Publication Practices May Distort Science” PLoS Medicine Oct 2008; 5(10): e201. (Published online Oct 7, 2008)
  13. John P. A. Ioannidis “Why Most Published Research Findings Are False” PLoS Medicine Aug 2005; 2(8): e124. (Published online Aug 30, 2005)
  14. Justin E. Bekelman, AB; Yan Li, MPhil; Cary P. Gross, MD “Scope and Impact of Financial Conflicts of Interest in Biomedical Research” JAMA. 2003;289(4):454-465..
  15. Kirby Lee, Peter Bacchetti, Ida Sim “Publication of Clinical Trials Supporting Successful New Drug Applications: A Literature Analysis” PLoS Medicine 5(9): e191. (Published online: September 23, 2008)
  16. How to recognize peer-reviewed (refereed) journals Angelo State University Library
  17. Pharmacorama’s “Methods for studying drugs”
  18. J. Bohannon “Who’s Afraid of Peer Review?” Science 342, 60-65 (2013) (Published 4 October, 2013)
  19. “Study Design 101″. The Himmelfarb Health Sciences Library; George Washington University November 2011
  20. “Study Designs”. Center for Evidence-Based Medicine; Nuffield Department of Primary Care Health Sciences; University of Oxford 2014

< Crazymeds’ Guide to Psychiatric Evaluations | Common Crazy Med Crap Index | Terms, Abbreviations, Acronyms & Initialisms
Why do they think they know so much? | About Crazymeds | Where does their money come from?

1 In which case you should probably call your doctor about getting your meds adjusted.

2 Similar to the media echo-chamber effect, where a piece of information, misinformation, or disinformation will appear in one of: a news organization's medium, an independent and respected (by the audience) blog, talk radio or podcast host, etc. It will then be picked up by one of the other listed outlets and reported as factual, regardless of if it is or isn't, but since they cited someone, that makes it seem legit. Then multiple news organizations of diverse media, more blogs, etc. disseminate it, with more cites, making it more legitimate, and so forth. That is how errors and lies become the truth, and why I've been saying "Everything is true" since 1994.

3 While it's true that 99% of the journals in PubMed et al. are biased toward the sort of medicine that involves people who received 10 years of training applying science-based treatments that are as profitable to Big Pharma as all those supplements at GNC are to their manufacturers - who are often subsidiaries of Big Pharma - weren't there a shitload of occupations all over the world regarding how 99% of the people agreeing about some important issue must mean they are right about it?

4 Those are all neurology-related journals. When my most-recent shrink retired he gave me 30-some psychiatric-related ones.

5 At the time she wrote the article she was Senior Lecturer at the University College London Medical School/Royal Free Hospital School of Medicine
Now she's Professor of Primary Health Care and Dean for Research Impact, Barts

6 Why is another story. I suck so much ass at noting why I updated something. Sorry.

7 I can understand wanting to try something out of a small study if you've been through 40 or more medications to treat your condition and your options are down to experimental surgery or homeopathy. But if you just don't like that you've gained five pounds over six months, you shouldn't even be reading PubMed in the first place.

8 If you read only the abstract it doesn't look like a particularly good study. I've read the whole thing, and many of the referenced studies are available. The evidence is far more damning than the abstract makes it out to be. And that was through 2002. Things only got a lot worse. Fortunately the increased transparency regarding funding and author conflict of interest seems to be working, as I'm seeing more drug company-funded studies resulting in failures. But the odds still favor the sponsor.

9 Hell, I could probably get published in Medical Hypothesis. Then again…

10 And it's going to be difficult enough to audit all the articles I've written to check for anything published by disreputable journals.

11 Mr. Beall seems to have a relationship with open-access journals that is similar to my relationship with open-source software.

12 Srsly? That got past everyone?!?

13 Someone usually almost always invariably asks the teacher "What good will this do us when we grow up?" Now you know.

14 Or his buddy Dick.

15 I'm sure there are people who think meta-analyses are more valuable than an open-label study. They probably publish a lot of meta-analyses.

16 Writes the guy whose Twitter feeds, Tumblr blog, and Facebook page are 90% retweets, reblogs, and shares. At least I do create content on this website and elsewhere.

17 While I've found many reviews to be more useful and, when referencing a couple of multicenter trials, far more accurate than single-blind studies, the fact that far too many reviews are paid for by Big Pharma drops their credibility.

18 But my standards are higher than many doctors', hence my links to smaller studies in the off-label uses sections of the drug guides.

19 The PK data from most Phase I clinical trials are useless. PK and safety testing almost always done on a small number of the usual suspects of human guinea pigs: healthy Caucasian men between the ages of 22 and 55. At least 80% of people who sign up to get paid for testing drugs are in that demographic (and this is pretty much the extent of stereotypical human drug testing). They take one, often sub-therapeutic dose of the med first thing in the morning on an empty stomach. The PK data from that trial doesn't always have any bearing on reality. Some drug companies, like Eli Lilly, use a much more diverse group of people, which is why you'll see lots of PK data for gender, age, and race on the PI sheets for Zyprexa and other Lilly meds. After a med has been on the market someone will run PK tests using people who take the med on a regular basis for conditions they have. Those are the data I try to use, and that is one time when an open-label study is vastly better than a clinical trial.

20 You know, like Shift Work Sleep Disorder.

21 My daughter was a participant in a phase IIIb study for a celiac medication. When she figured out she was in the experimental group she ate an entire pizza. She had an entire year to binge on gluten-containing food.

22 And a fine example of two pages that link to each other as a means of providing supporting data for each other. Like we have all over this house-of-cards of a website.

23 Think of it as a Zen koan, along the lines of:
The next sentence is true. The previous sentence is false.

24 As far as I'm concerned one is too many. Standards may vary.

25 The one exception being reputable online publications that are nearly at the level of peer-reviewed journals, and the article, or even the abstract, isn't available in a journal, PubMed, etc. I can understand how online publications can be limited to sources they want their readers to be able to reach. But that still should not be the only source to back up a treatment option claim.

26 A month. Really. I'm extremely lucky if I make more than $10 a month from Amazon sales.

Last modified on Monday, 19 October, 2015 at 19:33:41 by JerodPoorePage Author Date created Tuesday, 03 June 2014
“Evaluating Research Papers” by Jerod Poore is copyright © 2014 Jerod Poore Published online 2014/06/03
Citation options to copy & paste into your article:
Plain text:Poore, Jerod. “Evaluating Research Papers.” crazymeds.net. (2014).
with Microdata: <span itemprop='citation'>Poore, Jerod. "Evaluating Research Papers." <em>crazymeds.net</em>.(2014).</span>
Linked:<a href="http://scholar.google.com/citations?user=5rkux7sAAAAJ&hl=en&oi=sra">Poore, Jerod</a>. <a href='https://crazymeds.net/pmwiki/pmwiki.php/MedInfo/Researching'>"Evaluating Research Papers"</a>. <a href="https://crazymeds.net/pmwiki/pmwiki.php/Main/HomePage"> <em>crazymeds.net</em></a>. (2014).
with Microdata:<span itemprop='citation'> <a href="http://scholar.google.com/citations?user=5rkux7sAAAAJ&hl=en&oi=sra">Poore, Jerod</a>. <a href='https://crazymeds.net/pmwiki/pmwiki.php/MedInfo/Researching'>"Evaluating Research Papers"</a>. <a href="https://crazymeds.net/pmwiki/pmwiki.php/Main/HomePage"> <em>crazymeds.net</em></a>. (2014).</span>

All drug names are the trademarks of someone else. Look on the appropriate PI sheets or ask Google who the owners are. The way pharmaceutical companies buy each other and swap products like Monopoly™ real estate, the ownership of any trademarks may have changed without my noticing.

Page design and explanatory material by Jerod Poore, copyright © 2003 - 2015. All rights reserved.
Keep up with Crazymeds and and/or my slow descent into irreparable madness boring life. Pick your preferred social media target(s):

Almost all of the material on this site is by Jerod Poore and is copyright © 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, and 2015 Jerod Poore. Except, of course, the PI sheets - those are the property of the drug companies who developed the drugs the sheets are about - and any documents that are written by other people which may be posted to this site will remain the property of the original authors. You cannot reproduce this page or any other material on this site outside of the boundaries of fair use copying without the express permission of the copyright holder. That’s usually me, so just ask first. That means if want to print out a few pages to take to your doctor, therapist, counselor, support group, non-understanding family members or something like that - then that’s OK to just do. Go for it! Please. As long as you include this copyright notice and something along the lines of following disclaimer, I’m usually cool with it.

All rights reserved. No warranty is expressed or implied in this information. Consult one or more doctors and/or pharmacists before taking, or changing how you take any neurological and/or psychiatric medication. Your mileage may vary. What happened to us won’t necessarily happen to you. If you still have questions about a medication or condition that were not answered on any of the pages you read, please ask them on Crazy Talk: the Crazymeds Forum.
The information on Crazymeds pertains to and is intended for adults. While some information about children and adolescents is occasionally presented (e.g. US FDA approvals), pediatric-specific data such as dosages, side effects, off-label applications, etc. are rarely included in the articles on drugs or discussed on the forum. If you are looking for information regarding meds for children you’ll have to go somewhere else. Plus we are big pottymouths and talk about S-E-X a lot.
Know your sources!
Nobody on this site is a doctor, a therapist, or a pharmacist. We don’t portray them either here or on TV. Only doctors can diagnose and treat an illness. While it’s not as bad as it used to be, some doctors still get pissed off by patients who know too much about medications, so tread lightly when and where appropriate. Diagnosing yourself from a website is like defending yourself in court, you suddenly have a fool for a doctor. Don’t be a cyberchondriac, thinking you have every disease you see a website about, or that you’ll get every side effect from every medication1. Self-prescribing is as dangerous as buying meds from fraudulent online pharmacies that promise you medications without prescriptions.
All information on this site has been obtained from the medications’ product information / summary of product characteristic (PI/SPC) sheets and/or medication guides - which is all you get from sites like WebMD, RxList, NAMBLA NAMI, etc., the sources that are referenced throughout the site, our personal experience and the experiences family, friends, and what people have reported on various reputable sites all over teh intergoogles. As such the information presented here is not intended as a substitute for real medical advice from your real doctor, just a compliment to it. You should never, ever, replace what a real doctor tells you with something from a website on the Internet. The farthest you should ever take it is getting a second opinion from another real doctor. Educate yourself - always read the PI/SPC sheet or medication guide/patient information leaflet (PIL) that comes with your medications and never ever throw them away. OK, you can throw away duplicate copies, but keep at least one, as that’s your proof of purchase of having taken a med in case a doctor doubts your medical history. Plus they take up less space than a bottle, although keeping one inside of a pill bottle is even better.
Crazymeds is not responsible for the content of sites we provide links to. We like them, or they’re paid advertisements, or they’re something else we think you should read to help you make an informed decision about a particular med. Sometimes they’re more than one of those things. But what’s on those sites is their business, not ours.
Very little information about visitors to this site is collected or saved. From time to time I look at search terms used and which pages they bring up in an effort to make the information I present more relevant. And the country of origin, just because I’m geeky like that. That’s about it. Depending on how you feel about Schrodinger, our privacy policy should either assuage or exacerbate your paranoia.
Crazymeds is optimized for ridiculously large screens and browsers that don’t block ads. I use Firefox and Chrome, running under Windows 72. On a computer that sits on top of my desk. With a 23 inch monitor. Hey, at least you can make the text larger or smaller by clicking on the + or - buttons in the upper right hand corner. If you have Java enabled. Like 99% of the websites on the planet, Crazymeds is hosted on domain running an open source operating system with a variety of open source applications, including the software used to display what you’ve been reading. As such Crazymeds is not responsible for whatever weird shit your browser does or does not do when you read this site3.
No neurologists, psychiatrists, therapists or pharmacists were harmed in the production of this website. Use only as directed. Void where prohibited. Contains nuts. Certain restrictions may apply. All data are subject to availability. Not available on all mobile devices, in the 12 Galaxies Guiltied to a Zegnatronic Rocket Society, or in all dimensions of reality. Hail Xenu!

‘Everything is true, nothing is permitted.’ - Jerod Poore

1 While there are plenty of books to help you with hypochondria, for some reason there’s not much in the way of websites. Then again, staying off of the Internet is a large part of curing/managing the disorder.

2 Remember kids, Microsloth operating systems are like TOS Star Trek movies with in that every other one sucks way, way more. With TOS Star Trek movies you don’t want to bother watching the odd-numbered ones. With Microsloth OS you don’t want to buy and install the even-numbered ones. Anyone who remembers ME and Vista knows what I mean.

3 Have I mentioned how open source operating systems for commercial applications is one of the dumbest ideas in the history of dumb ideas?* I don’t even need my big-ass rant any more. Heartbleed has made my case for me. And that’s just the one that got all the media attention. The very nature of an open source operating system makes security as much of an illusion as anonymity on teh Intergoogles. Before you flip out too much: the domain Crazymeds is hosted on uses a version of SSL that is not affected by the Heartbleed bug. That’s one of the many reasons why I pay a lot of money and keep this site on Lunarpages.

* Yes, I know I’m using open source browsers. I also test the site using the now-defunct IE and Safari browsers. Their popularity - and superiority - killed IE and Safari, so that’s why I rely on the open source browsers. It’s like brand vs. generic meds. Sometimes the generic is better than the brand.

Enable Crazymeds’ Financial Solvency!

Enable Crazymeds to keep spreading our knowledge. Donate some spare e-currency you have floating around The Cloud.

Improve Your Social Media Skills


Follow our Highly Irregular Updates and Paranoid Rants Other News

Square this Circle

For Site News and NeuroPsych Research


Show us teh like™

Crazymeds: The Blog

For Site News and Crap that Distracts me from my Fucked-up Life

Crazymeds’ Tumblr

Mentally Interesting Advocacy

OpEd News

Daily Kos

Sites That Probably Suck Less Than Crazymeds

Crazymeds Merchandise

Available at Straitjacket T-Shirts

Vaccines Cause Immunity bumpersticker at Straitjacket T-Shirts

Stuck Up
All stickers $5. Now Available in Packs of 10 & 50

Mentally Interesting button at Straitjacket T-Shirts

Button It!
2.25″ $4 & 3.5″ $4.50. Now Available in Packs of 10 & 100