The 2014 Senate elections are not shaping up to be particularly favorable for the Democrats. While there are still scenarios where they could walk away breaking even, or even gaining a seat or two, those scenarios are pretty far-fetched. Current predictions vary somewhat, but seem to center around Republicans picking up somewhere between five and seven seats, with the overall range of possibilities a bit wider.
The nonpartisan explanations for this state of affairs have centered around three different factors: the president is unpopular, the president's party always loses seats in midterm elections, and the Democrats over performed in 2008, setting them up for a rough year in 2014 (you can see Bill Schneider making all three arguments here). In this article, I'll briefly discuss all three explanations, and then add a fourth.
Very little needs to be said about the first factor: The relationship between presidential approval and electoral outcomes has been thoroughly explored, and I have little to add. Likewise, the tendency of the president's party to fare poorly in midterm elections is so well-known as to require only an asterisk here: While the president's party has lost House seats in all but two post-World War II midterm elections (1998 and 2002), it has gained or broken even in Senate seats in five (1962, 1970, 1982, 1998, and 2002). That's somewhere between a third and a quarter of the postwar midterms, so our rule here is not really as "real" as it is for House elections.
Evaluating the third factor is a bit dicier. There are doubtless examples of elections where the president's party is so overextended from earlier wins that losses became almost inevitable: The large Democratic gains in 1986 were certainly a function of the huge Republican gains in 1980. Likewise, in 1958 the Democrats were able to gain 13 seats — the most seats won by a party since universal direct election of Senators began in 1914 — because Republicans had gained two seats in 1952, 12 seats in 1946, and four seats in 1940.
But we also have countervailing examples — including two of fairly recent vintage. In 2012, Democrats managed to gain two seats, their gain of six seats in 2006 (on top of gaining four seats in 2000) notwithstanding. Republicans beat the odds when they captured six seats in 2010 (not counting Scott Brown's special election win), despite the fact that Republicans hadn't suffered a net loss of seats with that Senate class since 1986. Big pickups for the parties in 1934, 1938, 1946, 1948, 1958, and 1986 were followed by relatively quiet elections six years later.
If we look at all Senate elections since 1914, it turns out that there actually is an inverse relationship between the number of seats that a party gains (or loses) and how that party fared in the elections six years earlier. The effect is just shy of full statistical significance (p=.06), but more importantly, it is small (b=-.28). If we confine ourselves to elections in the post-War period, the effect remains small (b=-.186), but is not significant (p=.297). If you look at the two prior election cycles, you get a stronger effect (b=-.37, p=.002), but even then, we would only expect Democrats to lose two seats this cycle based upon previous outcomes.
That actually sounds about right for this cycle. Let's look at this a different way. This table shows the 14 Senate races that are generally considered to be competitive this cycle: Two Republican seats and 12 Democratic seats. I've also shown the Democratic vote share in these seats in 2008, 2002, and 1996 (all figures are of the two-party vote, i.e., with third parties excluded):
What stands out is that, with respect to these particular seats, the Democratic vote shares are actually relatively stable. Alaska, Arkansas, and Montana are the only seats where the Democratic vote share has really fluctuated wildly over the past three elections for this class of senators. In the quite good Republican year of 2002, Democrats won half of these seats, and came close in another three (all three of which fell in 2008). In other words, the seats that are competitive this year are typically competitive.
When you look at it this way, Alaska stands out as the only seat Democrats are defending that is clearly the result of a fluke outcome from 2008. You can make a case for Minnesota, but it is a Purple state with a Blue cast, and that seat is really only on the outskirts of competitiveness right now. Stronger cases can be made for the Democratic wins in North Carolina and Louisiana — both of which probably depended on unusual youth and African-American turnout and the late break toward Democrats — but Democrats had been competitive in races for both of these seats for several cycles. Without splitting hairs too much, I think we can say that the Democrats' vulnerabilities this cycle are in part a function of their successes in the 2008 elections, but it is a small factor.
To get a better sense of what is going on here, we need to look at a fourth factor: call it "ideological drift." Table 2 shows these 14 seats, but instead focuses on the partisan index of the state. This measurement subtracts the Democratic or Republican two-party vote share in a state from the Democratic or Republican vote share nationally, allowing us to control for national effects (this is similar to Cook PVI, except that it looks at one presidential year, rather than two).
We're comparing the 2012 elections with the 1996 elections here — the Obama reelection with the Clinton reelection. Much attention has been paid to the leftward swing of certain states — Colorado, Nevada, and Virginia come to mind. This is reflected in the chart: We see moves leftward vis-à-vis 1996 in Alaska, North Carolina, and especially Colorado.
The countervailing rightward movements in other states, however, have received little attention. But they play a crucial role in understanding this election. Sixteen years ago, Arkansas, Kentucky, Louisiana, and West Virginia were substantially more Democratic than they are today.
Recall too that partisan indices only look at how vote shares for one candidate have moved, so these shifts probably convey an impression that the shifts were smaller than they actually are. The 19-point shift in West Virginia represents a movement from a Democrat winning, for example, 54% of the vote in West Virginia to winning 35% of the vote. So the net swing is the double the shift in partisan index — in this case an astonishing 38 points.
If we look at the three races where the Democrats are in the most trouble — West Virginia, South Dakota, and Montana — the playing field has worsened for them substantially since 1996. West Virginia has moved wildly, while South Dakota and Montana have moved from the outskirts of Purple state status to true Red state status.
Likewise, the two most threatened Democratic incumbents, Mary Landrieu in Louisiana and Mark Pryor in Arkansas, also hail from states that have swung strongly toward Republicans in the past two decades. The swing is probably even more pronounced in midterm elections, since they will not be able to count on replicating the surge in African-American voting their states enjoyed in 2008 and 2012. Finally, if Kentucky still had the same partisan orientation that it had in 1996, Mitch McConnell would probably be in very, very deep trouble.
Again, there are countervailing trends that help Democrats. But when we look at the summary numbers at the bottom of Table 2, this batch of states has moved from being a group with a midpoint that is roughly at the center of the country as a whole to one with a distinct Republican tilt.
So if we are trying to understand what is happening in 2014, we should definitely start with the president's approval rating, before moving on to the fact that the president's party tends to lose seats in midterm elections. We should look at the fact that 2008 was a good Democratic year, but it is also very important to keep in mind that there are demographic shifts that work against Democrats, and that they play a very substantial role in the party's precarious position this cycle.