A. Reasons for Doing Case-Level Analysis of Capital Federal Habeas Outcomes

So far, our analyses have considered the effect on rates of serious capital error of attributes of states and counties that use the death penalty—who lives there; how big a threat homicide is, and to whom; how well their law enforcement systems function; how well they support their courts and how much they burden the courts with capital and non-capital cases; how urbanized they are; what political pressures they put on the people who run their capital systems; and how often they use the death penalty. As we have seen, studying the relationship of capital error rates to the traits of the places, people and institutions that adopt and support the death penalty system, operate it, rely and make demands on it as a response to crime, and provide law enforcement alternatives to it, goes a long way towards explaining why the system makes more mistakes in some places than others.

Our final study, Analysis 19, asks a different question: What are the traits of the particular cases in which death verdicts are approved or reversed? Above, we discuss the ways in which addressing this question supplements our state and county analyses.⁶⁰⁴ Case-level analysis is extremely labor intensive, because it requires researchers to do all the data collection, as well as data analysis, themselves. There is no public or private source in the nation that systematically and comprehensively collects and distributes such information for even a single county, state or stage of review of capital cases. This is not surprising, given how difficult and painstaking the data-collection task is. The only comprehensive source of information (apart from the many informed participants at each capital trial) are the transcripts and files in each case. But those records run to thousands, or even hundreds of thousands, of pages in each case and are spread throughout tens of thousands of cities, towns and rural areas in the 34 states where death verdicts were finally reviewed during the study period. Gaining access to all those records for enough cases, states and years to provide an objective and statistically sound basis for comparison lies beyond the capacity of the research community as currently funded, and has never been attempted.⁶⁰⁵

The impossibility of tapping this first source of information forces researchers to rely on a second, less complete but more publicly and centrally available, source of information on each case: the published decision or decisions in which state and federal judges explain why they approved or reversed capital verdicts under review. Even here, the data collection task is enormous, given the more than 7500 lower, intermediate and high court decisions reviewing death verdicts during the 1973-1995 study period. And each such decision may contain hundreds of discrete items of information about the offense, defendant, victim, offense, lawyers, prosecutors, judges, procedures and the like, most of which, however, cannot be counted on to be reported in all or even most comparable decisions in other cases. Thus, apart from naming the defendant, appellate lawyers (with affiliations and locations), the trial judge or court, appellate judges, and (in most states) the aggravating and mitigating circumstances in the case, capital appellate decisions are not expected to report any uniform set of information about the case, but only the information the judge writing the opinion considers important enough to mention in deciding the legal issues he or she chooses to address. Moreover, many legal claims defendants present on appeal are not listed in published opinions, much less discussed. When capital verdicts are reversed, courts often address only the issue requiring reversal, because nothing more depends on the resolution of the other claims. In other cases, groups of legal claims are often decided summarily with phrases such as "all other claims were considered and rejected." Researchers can moderate this latter problem somewhat by reading every published decision at every review stage in every case. But doing so multiplies the number of decisions that must be read in each case—and is difficult in some states because of the absence of published state post-conviction decisions.⁶⁰⁶

Together with finite resources, three factors noted above led us to focus our case-level study on the federal habeas stage: the large number of verdicts reviewed at the state direct appeal stage, the far less complete information on cases that were reviewed at only one or the first two review stages, and the absence of published opinions in many state post-conviction cases. A fourth consideration is that our state-level regression analysis covering the federal habeas stage (Analysis 6) had the smallest number of data points to study. Although 598 final federal habeas decisions is a sizeable pool of cases when the outcome of each can be compared to that of the others, those cases provide a smaller basis for judgment in our state analyses, in which they are divided among 28 study states and 23 study years, leaving only 161 reversal rates for particular states and years to compare. That number is less than half the number of observations in our state-level analysis of the state post-conviction stage, and about a third the number of observations in our state-level analysis of the direct appeal stage. Fortunately, therefore, the most feasible focus for a case-level analysis is the review phase that is least comprehensively analyzed in our state-level analyses.

B. The Study

Analysis 19 studies a binary outcome: whether each of 598 death verdicts reviewed on habeas during the 1973-1995 study period was or was not reversed. We use logistic regression to see whether particular factors are significantly related to reversals of death verdicts on federal habeas review when other factors are considered at the same time.

We collected data on the 598 federal habeas cases from 1577 separate court decisions. As is indicated by the flow chart on p. 94 above, a federal habeas court generally is not permitted to review a death verdict that has not previously been reviewed and approved by at least one, and typically by several, state court decisions at the direct appeal and (in most cases) the state post-conviction stages of review.⁶⁰⁷ An average of about three decisions were read in each case, because we extracted information, not only from the final federal habeas decision, but also from every prior published decision in which a state court approved the same verdict on direct appeal or state post-conviction review and in which a lower federal court reviewed the verdict and either approved or reversed it. We gathered data on 11 categories of case-level traits—sentencing state and county; date and timing of the various milestones and procedures in the case; characteristics of the offense, defendant, victim, defense lawyers and state's attorneys at trial and on appeal, and judges at the various review stages; procedural history of the case; and legal claims and defenses. See Table 3, pp. 138-40, above. We supplemented this information with publicly available biographical data on practicing lawyers and judges, and with demographic data (race of defendant and victim, juvenile status, executions) from the Death Row U.S.A. publication described above.⁶⁰⁸

After collecting and checking the information, then coding and checking the coding, we identified the traits of particular cases on which we had information in most or all cases, and as to which there was enough variance to analyze.⁶⁰⁹ We then conducted simple bivariate tests of each trait to see if it, by itself, was correlated with habeas reversals of capital verdicts—meaning that the trait was present when death verdicts were reversed (or when they were approved) with sufficient regularity that there was only a small probability that the relationship appeared by chance.⁶¹⁰ Because this was a preliminary procedure for culling purposes, we provisionally retained traits if they were statistically significant or close to significance, or if logic or experience suggested they were important. (A trait that is not significantly related to reversals when considered by itself can turn out to be significant when considered with other factors.) Because some traits are not as important by themselves as they are in combination with other traits that play a similar role in capital cases, we created three indexes, or groups, of traits, used to measure how many traits of a particular type were present in each case:

• aggravating factors in the case that the state's statute expressly identified as a basis for a death sentence minus mitigating factors in the case that were expressly identified as such in a judicial opinion;⁶¹¹

• seven other factors about the defendant (prior criminal record, history of drug abuse, history of alcohol abuse, intoxication at the time of the offense and connection to the community where the crime occurred) and victim (gender and high or low status in the community) that while not always enumerated as aggravating circumstances in the relevant state statute, can have the effect of making the offense appear to be more aggravated;⁶¹² and

• the types of evidence introduced to prove guilt.⁶¹³

The final step was to measure the relationship of all potentially important factors at the same time, seeing which continued to be significantly related to federal habeas reversals of capital verdicts after accounting for the other factors. In comparing factors and sets of factors, we use several diagnostic tests:

• Is the factor's association with federal habeas reversals of capital verdicts statistically significant?⁶¹⁴

• How large is the factor's effect size, as indicated by the size of its exponentiated coefficient estimate?⁶¹⁵

  • The further a factor's exponentiated coefficient is above or below 1, the greater is the factor's effect size. The value is greater than 1 if an increase in the factor is associated with an increased probability of reversal, and less than 1 if an increase in the factor is associated with a decreased probability of reversal. Values for different factors cannot always be directly compared, because factors are scaled differently.⁶¹⁶

  • Effect size is predicted based on the assumption that other factors remain constant at their average.

  • Effect size is an estimate. In the tables below, we indicate the confidence interval for each effect size, which is the range of exponentiated coefficients within which there is a 95% probability that the actual effect size falls.

• How well do the outcomes (reversal vs. affirmance) that are predicted by the entire set of factors fit the actual 598 outcomes being studied?⁶¹⁷

Information on statistical significance, effect size (exponentiated B's) and fit is reported here. The regression tables are reprinted in full in Appendix G. Scores on some tests (e.g., fit) cannot be directly compared from one analysis to another, because they are sensitive to the number of cases being analyzed and because not all analyses study the same number of cases. For example, some federal habeas decisions are issued "per curiam," meaning the deciding judges are not named and that the political party of the President who appointed them cannot be determined. Those cases thus must be dropped from any analysis of the "party of appointing President" factor, making it inappropriate to directly compare the fit and related scores for that analysis to the scores for analyses in which the "party of appointing President" factor is not tested and more cases are studied.

C. Factors Related to High Probability of Federal Habeas Reversal: Low Quality State Court Proceedings, High Quality Lawyer, Weak Case for Death, Timing and Politics

As indicated by these tests, four analyses did the best job of explaining capital federal habeas reversals during the study period. Analysis 19A provides a useful benchmark because information on all factors it tests is available for nearly all cases in the analysis (595 out of 598), each of the significant factors it identifies is susceptible to a single, straightforward interpretation, and, among factors with those characteristics, it performs best on the various diagnostic tests. Each successive

analysis (Analyses 19B-19D) adds an additional, theoretically important and statistically significant factor, which seems to improve the overall power of the analysis, but covers fewer cases or is susceptible to multiple interpretations, or both.

1. Analysis 19A: Quality of proceedings and weak case for death.

Analysis 19A identifies the five conditions related to federal habeas reversals of capital verdicts that are listed in Table 17A below.⁶¹⁸ The p-value indicates significance. Effect size and its confidence interval are discussed above.⁶¹⁹ As is noted there, if the effect size is less than 1, an increase in the factor is associated with a decrease in the probability of federal habeas reversal; if effect size is greater than 1, an increase in the factor is associated with an increase in the probability of reversal.

a. Quality of court proceedings

Two significant Analysis 19A factors measure the quality of the proceedings that were used to determine the validity of the defendant's capital verdict, and a third tests the strength of the defendant's challenges to the verdict. The relationship of all three factors to reversals is as we predicted.

i. Low quality state court review procedure for finding the facts. If the state court review process that precedes federal habeas was of relatively high quality, given that before approving the verdict, the state reviewing court held a hearing and received evidence about its validity, the probability that the verdict will thereafter be reversed on federal habeas is lower. When the state court held a hearing and other factors are held at their averages, the probability of federal habeas reversal declines by over 40%.⁶²¹

Often, the validity of death verdicts turns on factual questions that are not answered by the record made at trial. Borrowing examples from four Supreme Court decisions reversing death verdicts, a verdict's validity may turn on:

• whether the defense lawyer incompetently failed to investigate the case before trial;⁶²²

• whether police officers had, but suppressed, evidence that the crime was committed by someone other than the defendant;⁶²³

• whether officials designed their method of calling possible jurors for trials to keep African-Americans from serving;⁶²⁴ or

• whether a state psychiatrist who examined the defendant before trial misinformed him about the purpose of the psychiatric interview in the process of eliciting damaging statements.⁶²⁵

Federal courts generally are not permitted to consider arguments such as these in favor of reversing death verdicts that were not first made to state courts,⁶²⁶ and must accept state court findings about what happened if the process the state court used to find the facts was reasonable or fair.⁶²⁷ When a challenge to a verdict raises a decisive and unresolved factual question, a reasonable and fair process for resolving the question usually requires an evidentiary hearing at which the state court hears testimony and other relevant evidence.⁶²⁸ Such hearings are not always required by state law, however, and state courts sometimes fail to hold them, lowering the quality of the state court review process and any findings it makes. Not surprisingly, that decrease in the quality of the state court review process increases the probability that a federal court will reverse a death verdict.

Together with results from the state and county analyses, this case-level result suggests that lower quality state court proceedings—as indicated by state courts' low funding,⁶²⁹ high caseloads of capital and other cases,⁶³⁰ and unreliable methods of finding out what happened—increase the probability that death verdicts will later be reversed due to serious error.

ii. High quality federal habeas lawyer. If the capital defendant's federal habeas lawyer probably provided high quality, well funded representation—as indicated by the lawyer's status as an out of state volunteer—it is more likely that a federal habeas court will find serious error and reverse the verdict. When this condition is present and other factors are held at their averages, the probability of federal habeas reversal increases by about two-thirds.

Most capital prisoners are too poor to hire their own lawyers and pay for needed investigation and expert DNA, psychiatric and other analyses.⁶³¹ Those prisoners thus are represented at the federal habeas stage by lawyers appointed or who volunteer to represent them and who must use public or donated funds to pay for investigators and experts. These lawyers fall into four categories, the first two including only in-state lawyers, the second two including mainly out-of-state lawyers:

• Lawyers appointed from the local private bar by federal courts before the federal proceeding has commenced. These lawyers almost always have little or no funding for investigators and experts apart from the resources the federal court provides.

• Lawyers employed by publicly funded capital defense offices that represent capital defendants sentenced by the funding state. These lawyers typically receive lower salaries than they could earn as private attorneys, but their offices sometimes employ investigators and have budgets for experts more generous than the funds federal courts otherwise provide.

• Lawyers in private for-profit law firms who represent capital prisoners on a volunteer, pro bono publico basis. These lawyers typically have high salaries and access to funds for use in employing investigators and experts.⁶³²

• Lawyers employed by private, national non-profit civil rights firms with capital punishment projects who represent capital prisoners across the country on a volunteer basis. These lawyers have fairly low salaries, but often have access to funds for investigators, experts and other support services.

The word "volunteer" in the last two categories refers to lawyers who are not chosen by the court to represent an indigent prisoner, but who instead volunteer their services to capital prisoners who retain them. Volunteer work is not entirely uncompensated. Most lawyers who volunteer are paid salaries or partner shares by their non-profit or for-profit firms using funds from other sources. Moreover, volunteer lawyers sometimes are partially paid by the federal court, which may appoint the lawyers on request by the capital defendant after the federal case is filed.⁶³³ Volunteer lawyers thus often supplement their own resources with modest amounts of government compensation and reimbursement for investigative, expert and other expenses.

There is no rating system for federal habeas lawyers, and the records from which we collected data provide little or no direct information on the quality of the lawyers in each cases. That information thus must be inferred based on the type of lawyer who represented the defendant. Quality is mainly a function of three traits—capital habeas and related experience, training, and resources available for litigating cases.

Experience. Most private lawyers, whether appointed (category 1) or volunteer (category 3), lack capital habeas experience. Public appointed lawyers eventually gain that experience but are subject to funding constraints and high turnover, so cases often are staffed by a single, inexperienced lawyer. Only private non-profit volunteer lawyers are routinely experienced or work on teams containing at least one experienced lawyer.

Training. Private in-state appointed lawyers typically have no special training or expertise as capital federal habeas lawyers. Although they often have experience as criminal trial lawyers, federal habeas cases are civil proceedings with many more similarities to complex civil litigation than to routine criminal trial work.⁶³⁴ Private volunteer lawyers do not always have capital expertise, but they almost always are experts in complex civil litigation. Moreover, their law firms have the resources to, and often do, pay for their lawyers to attend capital training conferences. And they also often seek advice from publicly funded or non-profit capital punishment projects, which, at the least (public offices) train their lawyers to do capital habeas work and, at best (private non-profits), employ highly experienced capital lawyers who conduct much of the available training for other lawyers from around the country.

Funding. The best funded of the four categories of lawyers are ones from private for-profit firms. Next are private non-profit lawyers, who adhere to professional standards calling for the provision of high-quality legal services but do not always have the funding to fully meet those standards. Then come publicly funded lawyers, who typically have high caseloads and stretched resources that can support only the basic services, but who sometimes employ investigators—an important extra resource. Private, appointed lawyers usually have only the base amount of resources.

Although federal habeas decisions rarely give the professional affiliation of federal habeas lawyers representing capital prisoners, they almost always list the city where the lawyers work. This is a useful indicator of the likely quality of habeas lawyers, because as the table below shows, there is a distinct tendency for relatively higher quality capital federal habeas lawyers to work out-of-state.

	Capital Habeas Experience?	Special Training/ Expertise?	High Resources?
In state (almost always):
Private Appointed	No	No	No
Public Appointed	Sometimes	Yes	Sometimes
Out of State (usually):
Volunteer, For-Profit Firm	No	Yes	Yes
Volunteer, Non-Profit Firm	Yes	Yes	Sometimes

Despite the indirectness of this measure of the quality of representation—which certainly misses some high quality, well-funded in-state lawyers⁶³⁵ and may include some low quality or poorly funded out-of-state lawyers⁶³⁶—the status of the habeas lawyer representing the capital prisoner is a powerful indicator of the probability of federal habeas reversal. Defendants who are fortunate enough to find an out-of-state lawyer are much more likely to have death verdicts reversed due to serious error than are other defendants. Evidently, the flaws in death verdicts under federal habeas review are serious and common enough—but also sufficiently hard to expose—that the quality of the appointed or retained lawyer matters a great deal.

Together with prior findings, these two Analysis 19 factors—that a state evidentiary hearing was not held, and that the defendant's federal habeas lawyer was from out of state—indicate that the quality of proceedings at each court stage is crucial: Low quality trials generate serious errors; low quality review procedures keep flaws from being found and corrected.

b. Weak case for death.

Analysis 19A's two remaining significant explanations for federal habeas reversals of capital verdicts reveal that the less aggravated or more mitigated a capital offense is by two distinct measures, the more likely it is that a federal habeas court will find serious error and reverse.

i. Low number of statutory aggravating factors minus the number of mitigating factors. As we note above, published court decisions in nearly all capital cases include a list of aggravating circumstances the sentencer formally relied on in sentencing the defendant to death.⁶³⁷ Typically, those circumstances are chosen from a list of qualifying aggravating traits of the crime or offender in the state's capital statute, and must be formally found to be present beyond a reasonable doubt. Generally speaking, at least one such trait must be present before a capital sentence may be imposed, with the sentencer then being required to consider all such circumstances in deciding whether to impose the penalty. Common aggravating factors are the defendant's prior history of violent crime, the fact that more than one victim was killed or threatened with death, the commission of other crimes such as robbery or rape at the time of the murder, the torturous method of killing, and a finding that the defendant is likely to commit violent crimes in the future.

Published decisions in most capital cases also list the mitigating circumstances the sentencing jury or judge formally found present in the case or, at least, those the defendant relied on as a basis for a sentence less than death that the reviewing court found present or found was supported by enough evidence that the sentencer could have relied on them. All mitigating factors supported by the evidence must be considered in deciding whether to sentence a capital defendant to die.⁶³⁸ Common mitigating circumstances are that the defendant was a juvenile at the time of the killing, acted under extreme emotional disturbance, had no criminal record, or could be rehabilitated.

Most capital statutes require sentencers to balance the aggravating circumstances against the mitigating ones in deciding whether to impose the death penalty, forbidding the death penalty if the mitigating circumstances outweigh the aggravating ones. The remaining statutes require the sentencer to consider both sets of circumstances at the same time, inviting some process by which the impact of one set of circumstances is discounted by the other set. Although this weighing or discounting process considers the quality of the aggravating or mitigating factors under the circumstances, and not just their numbers, the relative numbers provide researchers with the most objectively and consistently measurable estimate of the strength of the case for death—i.e., the degree of aggravation, or what we sometimes call "aggravating net of mitigation."⁶³⁹

Analysis 19A reveals that death verdicts federal habeas courts find seriously flawed and reverse tend to be ones where the case for death is weak. The stronger the case for death, the less likely it is that serious error is found and the verdict is reversed. Holding other factors at their averages, Analysis 19A predicts that with each decrease of one in the number of aggravating factors in the case, or each increase of one in the number of mitigating factors, the probability of reversal due to serious error rises by about 14%.

ii. Low number of other potentially aggravating factors. In our initial bivariate analyses, we found that seven other traits of the offender or victim correlated with federal habeas affirmances: that the defendant had a prior criminal record,⁶⁴⁰ abused drugs or alcohol, was drunk at the time of the offense, or committed the killing in his home community; or that the victim was a woman or had a high status in the community.⁶⁴¹ These circumstances all tend to make the offense seem more aggravated, given the defendant's bad behavior before or at the time of the crime, his offense against members of his home community or the victim's special vulnerability or place in that community. Although as we note below, it might also be appropriate to treat some of these circumstances as neutral (the victim's gender) or as mitigating (e.g., reduced culpability due to intoxication), each is associated with moral blame and has a capacity to aggravate the offense.⁶⁴² We accordingly combined the seven circumstances into an index, scored each case on the number of the factors present in the case, and tested the score as a possible explanation for capital federal habeas reversals.

On this measure like the previous one, the less aggravated the offense, the more likely it is that reversible capital error was found on federal habeas review. With each decrease of one in the number of the seven factors that is present in the case, the probability of reversal increases by about 24% in Analysis 19A, holding other factors at their average.⁶⁴³

iii. Summary: reviewing judges serve as substitute (but sometimes unreliable) sentencers. The significant aggravation-related factors support a common finding: Federal habeas judges tend to find serious, reversible error when the case for the death penalty is weak—i.e., when there are few aggravating factors relative to the mitigating ones. This finding has several implications.

First, the relationship between reversible error and weak evidence for a death verdict supports an interpretation offered above:⁶⁴⁴ Error leading to the reversal of death verdicts tends to occur in close cases where officials may be more tempted to cut corners to secure a death verdict, and where doing so is especially likely to change the outcome.

Second, this result provides additional evidence that the effects discovered in our prior state and county analyses operate mainly at the level at which capital-sentencing policy is set, not at the more local and particularized levels at which capital policy is implemented. Our state and county analyses tend to suggest that pressures to utilize the death penalty aggressively in response to high rates of serious crime and low rates of non-capital criminal enforcement lead to high error rates. This raises the question of the level at which those pressures operate. They might operate on particular cases, causing actors at relatively low levels of the official hierarchy—judges, lawyers and jurors involved in individual cases, but not the actors who set policy at the level of entire courts, prosecuting and defense offices and capital laws—to overreach or be careless in particular cases in which the pressure to punish is greatest. Or, alternatively, the pressures might operate at the level at which policy is set, causing state legislators, appellate and trial judges and district attorneys and police chiefs to write and interpret capital statutes so broadly or to design and implement capital procedures so carelessly that most or all capital cases in the jurisdiction are affected. In the former event, one would expect serious error to occur most often in cases with particularly aggravated offenses, because those are the cases where the pressures described above would be the strongest. In the latter event, on the contrary, one would expect serious error to occur when the case for a death verdict is the weakest—i.e., in cases that are not very aggravated, and were swept into the capital category only because the threshold for what counts as a capital case is set so low.

Our finding that habeas reversals tend to occur in weak cases for a death verdict supports the latter interpretation: Overarching death-penalty policy evidently invites officials in some states to use the penalty aggressively in marginal cases. The factually weaker the case for death is, the greater the need to overreach and commit errors in order that police can convince the district attorney to seek the penalty, the district attorney can convince the judge to allow it, and the state's case and court's instructions will convince the jury to convict and impose it.

Earlier, we addressed a similar issue, reaching a similar conclusion. We found that pressures to use the death penalty in response to crime are evidently related to the size of a state's black population relative to its general population, and to the risk of homicide to whites relative to the risk to blacks. We then asked whether these factors operate at the level at which capital-sentencing policy is set, or where policy is implemented, concluding that they operate at the policy-making level. It is not when individual cases are being tried, but when overall death-sentencing policy is set, that the two racial factors seem to generate pressure to overuse the death penalty, and thus to commit serious error. Once those pressures arise, they increase the chance of error in all cases in proportion to how weak the evidence for a capital verdict is, not just in cases with black defendants or white victims.⁶⁴⁵ Analysis 19 supports this interpretation: Neither the race of the particular defendant, nor the race of the victim, nor even a combination of the two, is significantly related to the probability of federal habeas reversal.

Third, the fact that federal habeas judges are more likely to approve death verdicts where overall aggravation is high suggests that appellate judges act as capital sentencers: They allow execution when the case is particularly aggravated, but bar it when aggravation is low. More crucially, the fact that reversible error is found so often—in 40% of all federal habeas cases, bringing the total reversed at all three court stages to 68%—suggests that federal reviewing judges act not merely as back-up capital sentencers in the occasional case in which an improper death sentence has slipped through, but as substitute sentencers, who determine the appropriate outcome in many cases. And that in turn indicates that the winnowing process at the trial stage—where it is supposed to occur—has failed.

Regrettably, there are three ways in which these substitute sentencers are not a reliable replacement for the real thing. As is noted above, legal rules generally bar federal habeas judges from reversing due to clear legal error unless it rendered the trial outcome unreliable.⁶⁴⁶ But a finding that the outcome is unreliable does not, by itself, require reversal. On the contrary, reversals are expressly forbidden where the outcome of trial was wrong but no legal error is found.⁶⁴⁷ As the examples in Part III show, one result of rules barring reversal where courts know the evidence is unreliable but no technical legal violation is found is the affirmance of death verdicts imposed on innocent people.⁶⁴⁸ In some cases, that is, appellate judges are poor substitutes for trial jurors, because legal rules keep them from fulfilling jurors' most crucial function: Barring execution where the evidence for a death verdict is weak.

In addition, reversal rarely bars retrial and a new death verdict. Although the hope is that retrial will lead to a lesser conviction, a less severe punishment or acquittal if the evidence turns out on retrial to be weak—as probably occurred in many of the retrials that we know ended in sentences less than death⁶⁴⁹—reversal does not assure the non-capital outcome that a properly functioning trial system would assure.

Lastly, the seven-circumstance index of supplemental aggravating factors that Analysis 19A finds significantly related to federal habeas affirmances includes some circumstances that might more properly be treated as either neutral (the victim's gender) or as mitigating (the defendant's drug or alcohol impairment at the time of the offense). That federal judges may mis-evaluate such circumstances suggests again that they are poor replacements for properly instructed jurors.

c. Factors that control for the strength of the federal habeas claim.

Analysis 19A finds that capital verdicts are more likely to be reversed by federal habeas judges who first granted the defendant an evidentiary hearing to inquire into one or more alleged flaws in the verdict than if no federal evidentiary hearing was held. Holding other factors at their averages, a verdict scrutinized at a federal evidentiary hearing is nearly 65% more likely to be reversed on federal habeas review than a verdict that is not scrutinized at such a hearing. In one sense, this factor bears out what we discovered above in regard to the quality of federal lawyers: The higher the quality of the federal proceeding—as indicated here by whether the court heard all the available evidence—the more likely reversal is. But because legal rules determine when federal judges may and may not hold federal evidentiary hearings, the factor also demonstrates a relationship between the reason evidentiary hearings are granted and both higher quality proceedings and the larger number of reversals they produce. We conclude that an important reason hearings are granted is the strength of the defendant's claim that his capital verdict is seriously flawed: The stronger the claim that the verdict is flawed, the more likely a federal court is to grant a hearing. And if a hearing is held, the federal proceeding is more likely to be reliable and to result in reversal. In the discussion that follows, we explain these conclusions and why it was important to include a factor that seems to prove the obvious: stronger claims for reversal more often lead to reversal.

As we have noted above, some claims tend to require proof of facts not already clear from the trial record, while other claims can be resolved based entirely on that record.⁶⁵⁰ As we also have noted, two claims that account for most federal habeas reversals usually require an evidentiary hearing to prove non-record facts: that the defendant's trial lawyer incompetently failed to find exculpatory evidence, and that police or prosecutors withheld such evidence.⁶⁵¹ Despite this link between evidentiary hearings and certain kinds of claims, the "federal evidentiary hearing" factor is not a measure of whether the capital prisoner raised one of these claims. This is because the vast majority of the 595 cases in Analysis 19A included claims of incompetent lawyering, state suppression of evidence, or both—most of which were denied because they were found to lack merit. Although incompetent lawyering at trial is the most common basis for habeas reversal, most such claims do not result in reversal. Only the strongest do—because reversible error must be serious error. Nor is there any significant correlation between reversal and having raised particular claims—or at least one claim—requiring an evidentiary hearing.⁶⁵²

Instead of linking the probability of reversals to the kind of claims raised, the "federal evidentiary hearing" factor appears to provide a rough estimate of the strength of the claims raised. The factor thus stands for the expected proposition that verdicts with stronger factual and legal grounds for reversal due to serious error are more likely to be reversed. Including such a factor in the analysis is useful, because, by controlling for the effect of the strength of the claims, the factor permits more confident conclusions that other significant factors have important explanatory power beyond the strength of individual claims.⁶⁵³

Federal evidentiary hearings are not legally required, and occurred in only 19% of the cases in our study. Typically, a federal habeas hearing will not be ordered unless four things are true:

• The state courts did not hold an adequate hearing.⁶⁵⁴

• The defendant asked for a hearing in state court, so it is not his fault that a state hearing was not held.⁶⁵⁵

• The claim based on which a capital defendant seeks a hearing is legally valid, meaning reversal is required if the facts the defendant wants to prove at the hearing are as he claims.⁶⁵⁶

• And the defendant can show that there is a reasonable chance that the witnesses he wants to call to testify and other evidence he wants to present will prove that the facts are as he claims.⁶⁵⁷

Item (1) depends on the quality of the evidentiary proceedings in state court. Analysis 19A already controls for that consideration with the "state evidentiary hearing" factor. Moreover, in the cases under study, there is no significant correlation between the fact that a state court held an evidentiary hearing and that the federal court did not.⁶⁵⁸ The "federal hearing" factor thus is independent of the "no state hearing" factor and reflects the other two conditions listed above: whether the defendant is at fault that no state court hearing was held, and the legal and factual strength of the federal claims. Because of the key role strong claims play in decisions to hold federal evidentiary hearings, because the stronger a claim is, the more likely it is that a federal court will conclude that the state courts' failure to hold a hearing was not the defendant's fault,⁶⁵⁹ and because federal courts during the study period sometimes exercised discretion to hold hearings even if the defendant was to blame for the absence of a state hearing but almost never held hearings on claims that were factually or legally weak,⁶⁶⁰ we conclude that the holding of a federal hearing is a fair indication of the legal and factual strength of the claims raised in the petition. This, in turn, suggests that the other significant factors in Analysis 19A are related to federal habeas outcomes independently of the effect of the strength of the defendant's claims.⁶⁶¹

2. Analysis 19B: The added effect of the passage of time.

Analysis 19B adds one factor to the five considered by Analysis 19A: the year the death verdict was imposed.⁶⁶² At a cost of only three cases dropped from the analysis because the published decisions did not say when the verdict was imposed, Analysis 19B performs better on the fit and other diagnostic tests discussed above. Results are summarized below.

Adding sentence-year in Analysis 19B does not alter Analysis 19A's results. All previously identified factors remain significant, with similar effect size. As for sentence year, Analysis 19B confirms what Analysis 6 already found: Controlling for other factors, death verdicts imposed later in the study period were significantly less likely to have been reversed on federal habeas by the end of the study period than earlier verdicts. As in Analysis 6,⁶⁶⁴ this result clearly occurs in part because of atypically long delays in deciding federal habeas appeals involving flawed verdicts. It is not, however, a reliable indicator of declining amounts of error.

During most of the study period, and all of its later years, federal habeas reversals of verdicts imposed in a given year took one to two years longer to occur than federal habeas affirmances of verdicts imposed in the same year.⁶⁶⁵ As of the study end date, therefore, a disproportionate share of verdicts from particular sentence years whose federal habeas outcomes were delayed beyond the study end date were flawed verdicts. On the other hand, verdicts that were finally reviewed by the end date were disproportionately without reversible flaws. Because many more later verdicts were awaiting review as of the study cut-off date than is true of verdicts imposed earlier in the study period, the bias in favor of counting unflawed verdicts but missing flawed ones has a larger effect on later than on earlier verdicts. So, even if every death-sentencing year has equal shares of flawed and unflawed verdicts, cases finally resolved by the study end date and thus counted in the study will include increasingly larger proportions of affirmances and smaller proportions of reversals for each successive sentencing year—leading the reversal rate reflected by the data to decline over time.⁶⁶⁶ At least the extent, therefore, and possibly the fact, that federal habeas reversals decline over time (taking other factors into account) is a reflection of the longer delays in reviewing flawed verdicts and not of the increasing quality of death verdicts over time.

Some, but not all, of the relationship between later verdicts and fewer federal habeas reversals may also be due to the fact that later verdicts reaching the federal habeas stage were less often flawed than earlier verdicts. Results from Analyses 3, 4 and 10 call for care in assessing this possibility. Those analyses much more accurately measure changes in the quality of death verdicts over time, and reveal that, after controlling for other factors, later verdicts reviewed on direct appeal—where about 80% of all capital reversals during the study period occurred—were found to be flawed more often than earlier verdicts.⁶⁶⁷ These results bar any confident conclusion that capital verdicts became less flawed over time after controlling for other factors. They could, however, suggest that federal habeas reversals declined over time (controlling for other factors) because state courts came to shoulder more of the burden of reversing flawed death verdicts, which remained stable or increased over time. Above we discuss evidence that state courts may indeed have done this to compensate for a series of legal changes that hurt the ability of federal habeas courts to cure serious capital error.⁶⁶⁸ This leaves us where Analyses 3, 4, 6 and 10 did: Taking other factors into consideration, state direct appeal courts found significantly more serious reversible error in later capital verdicts than in earlier ones.⁶⁶⁹ There is some evidence that state courts reversed more later verdicts than earlier ones to take compensate for a decreasing federal capacity to cure serious error on habeas.⁶⁷⁰ There is no clear and reliable evidence that, after controlling for other factors, capital verdicts had fewer flaws over time.⁶⁷¹

3. Analysis 19C: An additional measure of the strength of claims.

Analysis 19C adds another significant factor: the number of claims raised at the final federal habeas stage, which is another rough measure of the strength of the claims raised.⁶⁷² Although there is no indication of how many claims were raised at that stage in 34 cases, leaving 559 to be studied,⁶⁷³ the factor is worth considering as another means of controlling for the strength of the claim. Table 17C summarizes Analysis 19C's results.

Including the number of claims raised at the final federal habeas stage noticeably affects one significant factor from prior analyses: State evidentiary hearings are still negatively related to federal habeas reversals, but significance is now at the intermediate .06 level, and effect size drops some.⁶⁷⁵

The "number of claims" factor is significant and negatively related to reversal. The more claims raised, the lower the probability of reversal. Holding other factors at their averages, each additional claim raised at the final federal habeas stage is associated with a 14% drop in the probability of reversal. Federal habeas practice explains why this factor roughly reveals capital lawyers' assessment of the strength of their clients' best challenges to the death verdict.⁶⁷⁶

There is no limit on how many claims capital prisoners may raise in their petitions that commence federal habeas proceedings in the lowest federal court (the district court). Many district judges also permit briefs of any length in support of those claims. Things change dramatically once the lower court has ruled on a federal habeas petition. At that point, the losing party is permitted to appeal to a U.S. Court of Appeals, and thereafter, in rare cases, is granted permission to be heard on appeal by the U.S. Supreme Court. Because the first of these appeals takes place in the vast majority of capital habeas cases, the appellate decision is almost always the "final federal stage" at which we measured the number of claims raised.

If the capital defendant is denied relief at the lower court level, he may appeal to a higher court as long as the lower or higher court believes the appeal raises at least one substantial federal claim in support of reversal.⁶⁷⁷ At this stage, however, the rules tightly limit the number of pages the lawyer may file with the appellate court in support of the client's claims. This forces lawyers to choose between two strategies: Raise a small number of claims, devoting relatively many pages to a convincing argument on each. Or, raise a larger number of claims, giving appellate judges a wider array of possible reasons to reverse, but devote fewer pages to each argument. Over the run of cases, it is likely that lawyers will choose between these strategies based on the absolute and relative strength of the claims: If one or two claims are very strong, the best strategy is to focus on them, devoting as many pages as are needed to reveal how clearly they warrant reversal, while omitting weaker claims that might suggest a lack of confidence in the best claims. If the petition contains no particularly strong claim or claims, the preferred strategy is to choose a larger number of relatively weaker claims, thus maximizing the chance that each judge needed to form a majority will find at least one convincing claim. Over the long run, therefore, prisoner appeals raising a smaller number of claims are likely to include stronger claims than appeals raising many claims.

A similar effect arises when the capital defendant wins reversal at the lower level. In that event, the state may appeal. Because there usually is only a single basis for reversal, or at most a few, the appeal is limited to that small number of claims.⁶⁷⁸ Because a lower court judge's decision granting relief is a good indication of the strength of the small set of claims on which relief is granted, the small number of claims raised in most state appeals again is associated with the relative strength of the claims.

There are two important ways in which this indication of the strength of claims supplements the information provided by whether a federal evidentiary hearing was held. First, this indication applies to claims that do not, as well as ones that do, require factual development. Second, this factor mainly reflects an assessment of strength of claims by the capital defendant's lawyer (who decides how many claims to raise), not the federal district judge (who decides whether to hold a hearing). For these reasons, including both the "federal evidentiary hearing" and the "number of claims" factors gives us confidence that the other factors identified as important by Analyses 19A-19C (and 19D below) are important even after controlling for the strength of particular claims.

4. Analysis 19D: The role of federal judges' political affiliations.

Analysis 19D adds one final factor: whether a majority of judges who ruled on the case at the final federal habeas stage were appointed by Republican Presidents.⁶⁷⁹ This factor cannot be measured in 54 cases, because the judges responsible for the final federal habeas decision issued it "per curiam" (for the court), meaning they were not named in it.⁶⁸⁰ Including this factor in Analysis 19D leaves 535 cases to be studied⁶⁸¹ but appears to improve performance on the diagnostic tests detailed in Appendix G. Table 17D summarizes results.

Earlier we showed that most federal habeas judges on most federal panels that voted to reverse capital verdicts during the study period were appointed by Republican Presidents, which, given the Republican Party's strong stand against judicial interference with any but intolerably flawed executions, suggests the seriousness of the error federal habeas courts found.⁶⁸³ Supporting this suggestion is Analysis 19D's finding that judges appointed by Republican Presidents are in fact less likely to reverse death verdicts than judges appointed by Democratic Presidents. This relationship is significant at the .055 level and predicts a one-third decline in the probability of reversal if a majority of the deciding judges were appointed by Republican Presidents, holding other factors constant. (In an analysis in which sentence-year was omitted because of its sensitivity to delay, as opposed to more interesting explanations for a higher or lower probability of reversal,⁶⁸⁴ party affiliation of deciding judges was significant at the .01 level.)

The other factors considered in Analysis 19C remain significant in Analysis 19D and have approximately the same effect sizes, except that the effects of being represented by an out-of-state lawyer and receiving a federal evidentiary hearing increase. In this analysis (holding all other factors constant at their average as each factor is tested), the predicted probability of reversal:

• decreases between 10 and 15% for each successive sentence year, and each additional claim raised in the final federal habeas decision;

• decreases between 15 and 20% for each additional statutory aggravating factor net of mitigating factors, and for each additional supplementary aggravating factor;

• decreases by about a third if a state court held an evidentiary hearing, although this factor is significant at only the .08 level;

• decreases by about that same amount if most members of the final panel of reviewing federal judges were appointed by Republican Presidents;

• almost doubles if the capital defendant is represented by an out-of-state lawyer; and

• more than doubles if the defendant is afforded a federal evidentiary hearing.

D. Conclusion: Support for Prior Results Linking Reversals to Broad Use of the Death Penalty, Low Quality State Court Proceedings and Politics

Apart from the legal and factual strength of the defendant's claims (as indicated by whether a federal evidentiary hearing was granted and by the number of claims raised at the final habeas stage), four factors are reliably associated with an increased probability of federal habeas reversal:

• the trial jury's or judge's imposition of death for an offense that was not very aggravated;

• the low quality of the state review proceedings, as indicated by the denial of a state evidentiary hearing;⁶⁸⁵

• the high quality of the federal review proceedings, as indicated by relatively skilled and well-funded, out-of-state lawyers and also by the grant of a federal evidentiary hearing; and

• the political affiliations of federal habeas judges (judges appointed by Republicans being less likely to reverse).⁶⁸⁶

Analysis 19's findings about factors related to federal habeas reversals of death verdicts overlap a number of findings from our state and county analyses:

• The first Analysis 19 finding above echoes the most consistent finding of the state and county analyses: When jurisdictions and officials use the death penalty expansively, extending it to offenses that are not highly aggravated, the probability that verdicts they impose will be seriously flawed increases significantly.⁶⁸⁷

• Like some state and county analyses, case-level Analysis 19 links reversals to low quality state court proceedings—as indicated in other analyses by heavy capital and non-capital caseloads⁶⁸⁸ and low funding⁶⁸⁹ and here by a failure to use fair procedures to find facts (the second finding above).

• Just as high quality state review proceedings leave less serious error to be cured on later federal review, higher quality federal review is associated with more reversals due to findings of serious error. As the third finding above reveals, even after controlling for the strength of the claims presented—i.e., even among cases in which the factual and legal reasons for reversal are equally strong—higher quality federal review procedures and personnel are associated with the discovery and cure of more serious capital error.

• Three Analysis 19 findings—the first finding as applied to some of the seven supplemental aggravating factors, as well as the third and fourth findings—support another earlier conclusion: Appellate review is by no means a failsafe substitute for flawed capital sentencing at trial.⁶⁹⁰ On the contrary, federal judges:

• seem to treat some neutral and mitigating factors as aggravating ones;⁶⁹¹

• fail to reverse verdicts due to serious error they would have reversed if the indigent prisoner had been lucky enough to have a more skilled and better funded, out-of-state lawyer;⁶⁹² and

• are susceptible to political influences related to their affiliated political parties, tracking earlier findings that a fear of controversy may keep state court reviewing judges from reversing flawed rural and small town death verdicts,⁶⁹³ and that state post-conviction judges and to a lesser extent state direct appeal judges are susceptible to political pressures from judicial selection methods.⁶⁹⁴

• As does state-level habeas regression Analysis 6,⁶⁹⁵ Analysis 19 measures fewer reversals of later than earlier verdicts. In both analyses, however, this effect is at least in part a function of longer delays for habeas reversals than affirmances, which kept us from counting disproportionate numbers of habeas reversals by pushing them beyond the study end date. Given this bias against counting flawed verdicts, and given more reliable Analysis 3, 4 and 10 findings that direct appeal reversal rates are significantly higher for later than for earlier verdicts, Analysis 6 and 19 provide no reliable evidence that later verdicts are less flawed than earlier ones.⁶⁹⁶