The Chief Justice counted questions. He was asking the wrong one.

The rule holds up, and it still loses

In 2005, John Roberts counted the questions in 28 Supreme Court arguments. The losing side, he found, was almost always asked more. He published it with a joke: the secret to winning is getting the Court to ask your opponent more questions. Months later he was Chief Justice. I reran the count on every argued case since October Term 2005. The pattern is real. It still loses to a guess you can make before anyone speaks: the Court reverses.

Across 1,110 cases where we know who won, the side the bench spent more words on lost more often than it won. I counted it six different ways: turns instead of words, dropping the amicus sections. Every version still beats a coin flip.

So Roberts saw something true. The bench really does lean on the eventual loser. Can you use that to pick winners? That takes a number his paper never mentioned.

Any prediction has to beat the guess you’d make knowing nothing. That free guess is called the base rate, and here it’s lopsided. A petitioner is at the Court because a lower court ruled against them and the justices agreed to take another look. They could have said no. Saying yes isn’t neutral: by the first minute of argument, the petitioner is already the two-to-one favorite.

Always guessing the petitioner calls ~66% of these same cases. Counting questions calls ~61%. Run the two head to head, case by case, and the free guess wins by a margin too big to be luck.

That’s the bar I use for everything here, including my own model. A rule can detect something real and still be worth less than the free guess. “Almost always” sounds like plenty until you know what guessing gets you.

“Who wins” is one number per case. The Court doesn’t give you one number. It gives you nine.

Every vote answers two questions

I scored every argued case since October Term 2005. Transcripts come from Oyez, which publishes the audio and the speaker-labeled text. Outcomes and votes come from the Supreme Court Database.

Any single vote can be scored two ways.

The first is the one people argue about. Was this a conservative vote or a liberal one? Somebody has to decide that. Spaeth, Epstein, Nelson, Martin, Segal, Ruger, and Benesh code each decision against a published rulebook: a vote for the criminal defendant is liberal, a vote for the business against the regulator is conservative, and so on. It’s a judgment call. Every ideology number below rests on it.

The second question sounds like paperwork. Which of the two lawyers in the room did the justice side with? No interpretation is needed, because the Court’s own judgment says who won.

Throwing out a lower-court ruling isn’t conservative or liberal by itself. It depends entirely on what that court did. The same data answers the two questions in opposite ways.

The exhibit below is every vote I have, one box per justice per argued case, twenty years wide. Rows are justices and columns are cases in the order they were heard. Use the control to switch between the two questions.

Score by ideology and the bench sorts into two horizontal blocks you could have drawn from memory. Switch to which lawyer each justice sided with and the blocks dissolve into vertical stripes. A vertical stripe means a whole column voted the same way, which is what near-unanimity looks like. The only thing that changed is the question.

Who appointed a justice predicts the label, not the vote

If you want to guess how a justice votes, the free answer is the president who appointed them. It’s in every news story about the Court, usually as the whole explanation. If you’re asking how the vote gets labeled conservative or liberal, it belongs there. Across 7,956 votes in cases with at least one dissent, party calls the ideological direction ~69% of the time, against 50% for a coin flip.

That’s a count of how the Database coded the votes. When I get to the argument scores, those come from a model.

Every ideology figure here uses only those divided cases, because a unanimous decision gives nobody a chance to break from a label.

People treat that label as if it got stronger over twenty years. It didn’t. Two justices left. The people who stayed didn’t get more predictable. Take John Paul Stevens and David Souter out of the average and the climb is gone. No sitting justice drifted toward their label. The bench was replaced.

Stevens was appointed by Gerald Ford, and in divided cases the label built from that fact was right ~19% of the time. A coin flip is 50%. Souter was appointed by George H. W. Bush and is in the same place. The label pointed the wrong way.

The argument reads one justice

Now the other question: which lawyer will each justice side with? I let four things compete: nothing at all, the appointing party, the justice’s own voting record, and the argument.

“The argument” means four measurements taken off the transcript, and no names, parties, or votes. Which lawyer the justice spent words on, how far that lean sat from the rest of the bench that day, how much the justice talked relative to their own habit, and how fast. The model doesn’t see the Court’s own decision. It only sees the transcript.

Every score below is out of sample: each vote is called by a model trained only on earlier terms, so the model has never seen the vote it’s predicting. A model graded on cases it studied would just be memorizing.

Ask which side a justice backs. Party, the justice’s own record, and a guess that knows nothing all land on ~62%. That’s the base rate again. Knowing who appointed you says nothing about whether you’ll throw out a Ninth Circuit tax ruling. The argument gets ~65%.

That’s only a few points, so I wanted to know how firm it was. I scored them with a stricter measure, one that punishes confident mistakes hardest, and reshuffled the same 1,201 cases. The argument beats the justice’s own record every time.

Ask the ideology question and it goes the other way. The president who appointed a justice and the entire oral argument land on the same number. An hour of listening tells you nothing a Wikipedia page wouldn’t. Adam Feldman reached the same split independently on a single term while I was fitting this. His numbers are in Sources.

That’s the average across the whole bench. Look at justices one at a time and some of them break it. Test 15 justices one at a time and roughly one will look remarkable by pure chance. So I set the bar higher, high enough that the whole set of tests still has the usual odds of a false alarm. Only Stevens, Gorsuch and Thomas clear it. The other bars in the exhibit are noise.

Gorsuch’s questions predict his vote at ~76%, against ~60% for his own voting history.

That margin isn’t one strange term. The room reads him better than his record in seven of nine terms.

Clarence Thomas goes the other way. Once he speaks, his questions point away from his vote.

The third one who clears the bar is John Paul Stevens, and the gap looks bigger than it is. His own record is close to unbeatable because he voted one way almost every time. Stevens wasn’t hard to read. He was easy to predict for anyone who watched him instead of reading his résumé.

I can’t tell you why the room works for Gorsuch and not the other 14. My reading is that he argues his way toward a position in public, so a measure of where his attention goes tracks something real. But that’s me interpreting a number. I can’t test it.

Add the nine votes together and the edge disappears. Across 1,201 cases, the model and the one-word guess “the Court reverses” land within a point of each other. Run both rules on the same cases and you can’t tell them apart.

If you want the likely winner, the best clue arrives before the argument begins. If you want to know how a named justice is leaning, the argument is the only thing I tested that helps.

I published predictions for 22 undecided cases from October Term 2025, timestamped before the outcomes, in the forecast file. If a case sits near 50%, the model has nothing to say. That’s not the same as the case being a toss-up.

The pattern is real. It starts too late.

Roberts saw a real pattern. You still can’t use it to pick winners. By the time anyone asks a question, the Court has already tipped the odds by agreeing to hear the case.

We do the same thing with the party label. We talk about justices as if the president who appointed them is the whole story. Stevens voted liberal for thirty years in public and the label survived anyway.