Grade a claim the book never graded
This register grades seventy-nine claims. You have almost certainly met an eightieth: something in a group at two in the morning, something a kind person sent you, something with a name this book has never printed. This page runs that claim through the same procedure the book used on the other seventy-nine.
What comes out is the most that claim could be graded on what you are holding. Every question below can lower that and none of them can raise it. There is nothing you can answer that will make this page more encouraging than the evidence behind the claim already is.
It cannot see you. It is never told what the compound is, it does not know your diagnosis or your treatment, and there is no answer you can give it that will make it say anything about whether something would help you. What it produces at the end is a question for the people treating you.
The answer is usually D, and that is an answer
Five of the six ways of answering the first question end the procedure immediately, at D. That is not the page failing to grade your claim. Most claims about most compounds have no human study behind them, and D is what the book says about every one of them. A page that found something more encouraging to say would be inventing it.
The whole procedure, written out
The tool above is a shortcut through these tables. With this page printed and a pen you reach the same answer, and the four questions in the middle are the evidence ceiling run by the same function the register runs against itself.
| If your answer is | Then | Because |
|---|---|---|
| A study in people, published, that I could find and open | No change | Continues to the questions about the study itself. |
| A study in animals, or in cells in a dish | Ends here, at D | A finding in animals or in a dish is a reason to do the study in people. It is not a small version of that study, and most things that work in a mouse do not work in a person. That is the ordinary result rather than bad luck. |
| What happened to one person, or to several people, told as their stories | Ends here, at D | The people it did not work for do not write the post, and the people who died cannot. A set of accounts is selected by its own outcome before you ever see it, and there is no way from inside it to know how many are missing. Cancers also behave unpredictably in individuals on their own, which is what makes one recovery impossible to attribute. |
| A clinic, a shop or a practitioner who provides it | Ends here, at D | That does not make the claim false and it does make the source the wrong one to settle it. A source that reports a finding and a source that sells the thing the finding is about are two different kinds of source, and only one of them loses anything by being wrong. |
| An article, a video or a post that does not name a study | Ends here, at D | A report that does not name its study cannot be checked against it. You are not holding weak evidence; you are holding a description of evidence you have not seen, and the two differ more often than they agree. |
| Nothing I can point at | Ends here, at D | This is the most common honest answer about any compound, and it is a real answer rather than a gap in this page. It does not say the claim is wrong. It says that at this moment nobody can tell you either way, and that anyone who tells you either way is going beyond what exists. |
Second to fifth questions: the study itself
Only reached if the first answer was a study in people. These four are the evidence ceiling, and every answer they can give is on that page in full. They cap the grade; they never lift it.
| If your answer is | Then | Because |
|---|---|---|
| The study itself, or its abstract | No change | This answer does not lower the grade. |
| A report that named the study, so I could find it | No change | This answer does not lower the grade. |
| A report that did not name the study | Nothing above C | Between a paper and a headline about it, something is almost always lost, and what is lost is usually the qualification. Nothing above C can rest on a version of a study you cannot compare with the study. |
| Somewhere that provides the thing | Nothing above D | A study can be real and its account can still be shaped by who is retelling it. Where the teller provides the thing, the retelling cannot carry the claim, whatever the study was. |
| If your answer is | Then | Because |
|---|---|---|
| Yes, it measured the thing the claim is about | No change | This answer does not lower the grade. |
| It measured something else, related or not | Nothing above D | This is the commonest way a strong study ends up standing behind a claim it does not support, and it lands at D rather than somewhere in the middle because the middle would be false. A trial that measured tiredness is good evidence about tiredness and no evidence at all about survival, however large and however well run. The register grades exercise on six separate claims for exactly this reason, and moved one of them down when the source turned out not to report the endpoint the claim was about. |
| I could not tell what it measured | Nothing above D | This is not a mark against you or against the study. It is that a claim can only be supported by a measurement somebody can name, and treating an endpoint you could not find as though it were the right one would be filling an absence in with what seems likely. That is the one thing this register never does. |
| If your answer is | Then | Because |
|---|---|---|
| Yes, a different group has found the same thing | No change | This answer does not lower the grade. |
| No, this is the only one | Nothing above B | Studies that find something get published and studies that find nothing often do not, so the first result on any question is the one most likely not to hold. A single finding is a reason to look again rather than a settled one. |
| I do not know | Nothing above B | Not knowing whether anyone has repeated it is not the same as knowing they have, and the register does not fill an absence in with what is typical. If you can establish that somebody has, come back and say so. |
The eighth question has one exemption. A large trial that randomly assigned people and measured something that happened to them is not held back for standing alone: two of the strongest entries in this register rest on exactly that, and a rule here that the register does not follow would be teaching something false.
What each answer means, and what would change it
- Grade A — Randomised human evidence of benefit.
Nothing in this procedure lowered it, which means the evidence you are holding is as strong as the procedure can recognise. It still says nothing about you, your diagnosis or your treatment, and the person who can join those up is the person treating you.
A question worth carrying to your team: I have read about something with what looks like randomised evidence behind it, so would it be reasonable for someone to look at whether it has anything to do with my situation?
- Grade B — Good human evidence, with a limit worth knowing.
What would raise this is the limit named above being removed by somebody: the trials being put in front of you, a second group finding the same thing, or a design that assigns people rather than following them.
A question worth carrying to your team: I have read about something with human evidence behind it that has a limit I can see, so is that limit the kind that matters here?
- Grade C — Human evidence exists and does not settle it.
What would raise this is a study that assigns people rather than watching them, or a report you can check against the study it describes.
A question worth carrying to your team: I have read about something where human evidence exists and does not seem to settle it, so is it worth anything in my case?
- Grade D — No human evidence of benefit, or none that survived testing.
What would raise this is a completed study in people, reporting the thing the claim is about. Until one exists there is nothing here to grade upward, and saying so is the honest position rather than a cautious one.
A question worth carrying to your team: I have read about something where I could not find human evidence of benefit, so is there anything known about it that I should hear?
What this page is not
It is not a check of a compound. It never learns which compound you had in mind and the answer would be the same if you changed it, because what is being graded is the evidence you found rather than the substance it is about. It is not a second opinion, it is not a reason to start or stop anything, and the grade it produces is a ceiling rather than a finding: a claim that could be graded A may still be one nobody has shown to work.
Every claim the book does grade is in the grade table, and the ladder itself is on the front page.