Fan Satisfaction Index End of Season Bball Survey

Fan Satisfaction Index End of Season Bball Survey

Submitted by OneFootIn on April 2nd, 2018 at 11:42 PM

It might feel a bit too soon to share your feelings, but science doesn't stop for your pain or suffering.

It's time for the last basketball fan satisfaction survey.

How are you feeling about the season with all things said and done?
How are you feeling about next year?

Take the survey here: https://goo.gl/forms/vaO1zAH11UwEK5Kh2

Fan Satisfaction Index: BBall Regular Season Results

Fan Satisfaction Index: BBall Regular Season Results

Submitted by OneFootIn on March 1st, 2018 at 12:25 PM

Solid. Surprising. Ascending. MAARvelous. Antrorse.

These are just a few of the words fans used to describe the basketball team’s regular season. Sitting at 24-7 after throttling Maryland last Saturday, Michigan fans were in a good mood, felt very satisfied about the regular season, and displayed a great deal of optimism about Michigan’s chances in the NCAA tournament.

It should come as little surprise to learn that a decisive road win against a team that Ken Pom and others expected to beat Michigan induced a serious case of satisfaction. Satisfaction with the game averaged 94.3 and as Figure 1 shows, responses clustered in an unusually tight range (except for one person who I think might have accidentally typed ‘10’ when he or she was trying to type ‘100’).

Figure 1. Maryland Game Satisfaction
(X axis is game satisfaction on a 0-100 scale; Y axis is # of respondents)

Season satisfaction was also high after the Maryland game, averaging 85.7 with a standard deviation of just 6.8, again showing a remarkable level of uniformity of opinion. To some degree the level surely reflected (as was the case during the football season) the flush of positive emotion that comes with a victory. At the same time, fans have every reason to be satisfied with the season viewed in broader perspective.

Sure, the season could have been really special had they not blown a couple of golden opportunities (Purdue twice, OSU, etc.) and lost a couple stinkers (Nebraska, Northwestern). On the flip side, however, how many fans had confidence at midseason that Michigan would wind up 24-7 and look this dangerous as it heads into the postseason? Given how long it took Michigan to figure out who was going to play what roles and for various pieces to round into form, their record seems like a major achievement, not to mention a great sign of things to come.

Figure 2. Regular Season Satisfaction
(X axis is season satisfaction on a 0-100 scale; Y axis is # of respondents)

This survey offered fans the opportunity to share the single word that they felt best described the season. In true MGoBlog fashion we received several creative responses. The word cloud below offers a visual summary of the responses with the most popular responses appearing as the biggest words in the cloud.

Figure 3. Describe the Season in One Word

The final question on the survey asked fans to predict how far Michigan would make it in the NCAA tournament. Riding a five game winning streak, including a smack down of the Buckeyes and two road victories against dangerous opponents, the fans are feeling bullish. A majority will clearly be disappointed if Michigan isn't playing on the second weekend. 

Figure 4. How Far Will Michigan Go?

  Percent n
1st Round 0.4 1
2nd Round 10.4 24
Sweet 16 54.6 126
Elite 8 25.5 59
Final Four 3.5 8
Championship Game 1.3 3
They'll Win It All 4.3 10

 

Fan Satisfaction Index: Outback & End of Season Results

Fan Satisfaction Index: Outback & End of Season Results

Submitted by OneFootIn on January 15th, 2018 at 10:50 AM

Quick note: For those unfamiliar with the FSI, it is a weekly survey asking fans to rate their feelings about each game and the season so far on a 0-100 scale. To catch up check out my blog here: http://mgoblog.com/diaries/onefootin

Who has it better than us? Well, according to my calculations, more than half of the Big Ten has it better right now. And I’m going to bet you won’t like who’s on top.

Let’s take this in two parts.

The Outback Bowl

First, there was that bowl game. As Figure 1 makes clear, this game felt bad. In fact, at a satisfaction level of 17.6 on our 0-100 scale, it felt worse than every regular season game except the Michigan State game.

This isn’t too surprising. It was bad enough to lose when favored by 7 points against an uninspired-looking South Carolina team that had just fired its offensive coordinator. It got worse when Michigan, leading 19-3, managed to fumble at the 5. It bottomed out when it turned out that was just the beginning of the second half Errorpalooza. Watching Michigan self-immolate while the Gamecocks scored 23 unanswered points was deeply aggravating, to put it mildly.

Figure 1: Outback Bowl Game Satisfaction.

(On a scale of 0 to 100, where 0 is the worst you ever felt after a game and 100 is the best you ever felt after a game, where would you rate your feelings about the Outback Bowl?)

X-axis is game satisfaction and Y-axis is # of respondents

Adding insult to injury, the loss to the Cocks took most of the remaining mojo from the fan base regarding the season as a whole. Season satisfaction clocked in at 24.9 – its lowest point of the season. 8-5 doesn’t feel good, as it turns out.

Figure 2: Season Satisfaction after the Outback Bowl.
(On a scale of 0 to 100, where 0 means the season went horribly and 100 means the season went perfectly, how do you feel Michigan's season went?)

X-axis is season satisfaction and Y-axis is # of respondents

Calculating B1G Fans’ Season Satisfaction

Okay, now for part two. Michigan’s season was unsatisfying but perhaps – out of a morbid sense of curiosity – you are wondering how Michigan fan satisfaction stacks up against other fan bases around the league.

Modeling Satisfaction from Our Data

Since I did not survey non-Michigan fans directly I used a regression analysis of our Michigan fan data to come up with a formula for calculating satisfaction for other fan bases. This approach comes with clear limitations. First, since we only have one season of Michigan data we don’t even have a perfect model of how Michigan fans will react to all situations. Just to take a couple of examples, we have no data on how fans respond to an unexpected victory over a ranked opponent, nor any idea how season satisfaction would look during a season where Michigan outperformed overall expectations. For that reason, our regression model is certainly far from perfect.

Second, even if our model were perfect for Michigan fans, it is very likely that other fan bases would react somewhat differently to the same situations. Given historical circumstances (spoiler alert!), Purdue’s fan base is likely to be happier with a 7-6 record on the season than Michigan’s is with 8-5. And though all teams have rivalries, we probably shouldn’t assume that all fans feel the same about them. I am pretty convinced, for example, that Sparty and Buckeye fans get more satisfaction from beating Michigan than the other way around.

With these caveats in mind, I still think we can provide a pretty reasonable estimate of B1G fan base satisfaction based on how Michigan fans responded during the season. For Michigan fans, based on 2605 responses over 13 games, the basic equation for game satisfaction is: 49.63 + (1.03 x Margin of Victory/Defeat) + (0.28 x Margin vs. Vegas) – (20.8 x Surprise Loss).

Margin of Victory/Defeat, clearly, is just measured by how many points more/less Michigan scored than its opponent. This captures both whether a game is a victory or defeat as well as its intensity. Margin vs. Vegas is how many points more/less Michigan scored than its opponent relative to the Vegas line. This captures general fan expectations about how the game went, which as we have discussed in past weeks is a critical component of how people feel about the outcome of a game. Surprise Loss is a variable I threw in because it was clear that unexpected losses – i.e. where Michigan was favored to win by Vegas – hurt more than usual.

In English, the model assumes satisfaction is about 50 points on our 100-point scale and then slides things up or down based on whether Michigan won or lost, by how much, and by how much relative to expectations. An additional point of margin in a victory adds about one point to fan satisfaction (vice versa for a loss). For every touchdown by which Michigan beats the Vegas spread you can add another 2 points of satisfaction, while a surprise loss sucks about 21 points of satisfaction from the average fan.

According to the magic of statistics this formula explains 70% of the variation in individual game satisfaction ratings. In the land of predicting individual opinions, 70% is pretty darn good, especially since all we have is data about the games and we don’t have any information on the respondents (Imagine, for example, trying to predict presidential popularity from economic conditions but without any information on respondents’ political affiliations). 

Table 1 below illustrates how well the formula does predicting the typical fan’s satisfaction compared to the average satisfaction measured by the survey for each game. Though the predicted satisfaction misses big in a couple cases, overall it tends to come pretty close, with an average absolute difference of less than six points across all 13 games. After a few more seasons worth of data the predictions should get better.

Table One. Real vs. Predicted Michigan Fan Game Satisfaction

Game Actual Sat Predicted Sat Actual - Predicted
Florida 80.9 74.5 6.4
Cincinnati 59.9 65.3 -5.4
Air Force 62.9 61.2 1.7
Purdue 76.5 71.3 5.2
Michigan State 17.5 14.9 2.6
Indiana 51.6 56.5 -4.9
Penn State 23.9 6.1 17.8
Rutgers 73.9 69.5 4.4
Minnesota 78.5 78.6 -0.1
Maryland 73.5 81 -7.5
Wisconsin 28.8 30.7 -1.9
Ohio State 27.7 39 -11.3
Outback Bowl 17.6 11.5 6.1
    Average diff 5.8

 

 

 

 

 

 

 

 

 

 

 

 

The formula for season satisfaction is pretty similar. If you’ve been reading the diary this season you know that the average fan’s sense of the season is heavily tied to the game they just watched. As a result, assessments of the season varied a lot more on a weekly basis than they probably should have based strictly on the amount of new data coming in each week. The other significant variable in the season satisfaction formula, unsurprisingly, is the number of cumulative losses. Nothing says satisfaction like winning; nothing destroys it more than losing.

As a result, our season satisfaction formula after the 2017-18 season looks like this: 29.84 + (.62 x Game Satisfaction) – (3.388 x # Cumulative Losses). This model explains 73% of the variation in individual season satisfaction assessments over the 13 games of the season. Again, not too shabby. Table Two provides the summary.

Table 2 Real vs. Predicted Michigan Fan Season Satisfaction

Game Actual Sat Predicted Sat Actual - Predicted
Florida 85 80 5
Cincinnati 77.2 67 10.2
Air Force 72.7 68.8 3.9
Purdue 76.7 77.3 -0.6
Michigan State 40.5 37.3 3.2
Indiana 53.7 58.5 -4.8
Penn State 33.7 37.9 -4.2
Rutgers 62.9 68.9 -6
Minnesota 69.1 71.7 -2.6
Maryland 69.9 68.6 1.3
Wisconsin 36.3 37.5 -1.2
Ohio State 36.8 33.5 3.3
Outback Bowl 24.9 23.8 1.1
    Average diff 3.6

 

 

 

 

 

 

 

 

 

 

 

 

Who Has It Better Than Us? Season Satisfaction across the Big Ten

If you’re still with me, Table 3 brings home the sad fact: Michigan’s implosion in the Outback Bowl, combined with its five losses on the season, put Michigan fan satisfaction below all seven B1G teams that won their bowl games and even below Indiana, which lost to its rival Purdue to end its season.

Table 3 End of Season Fan Satisfaction in the B1G

Team Season Sat Record (Ranking) Final Game (Game Sat)
MSU 70.2 10-3 (15) Beat #18 WSU 45-17 (81.5)
OSU 65.9 12-2 (5) Beat #8 USC 24-7 (69.1)
Wisconsin 63 13-1 (7) Beat #10 Miami 34-24 (61)
PSU 59 11-2 (8) Beat #11 UW 35-28 (58)
Purdue 56.1 7-6 Beat Arizona 38-35 (75.2)
Northwestern 50.1 10-3 Beat Kentucky 24-23 (49.1)
Iowa 49 8-5 Beat Boston College 27-20 (58)
Indiana 31.4 5-7 Lost to Purdue 31-24 (40.7)
Michigan 24.9 8-5 Lost to South Carolina 24-17 (17.6)
Minnesota 14.9 5-7 Lost to Wisconsin 31-0 (14.2)
Rutgers 9.5 4-8 Lost to MSU 40-7 (10.9)
Nebraska 2.74 4-8 Lost to Iowa 56-14 (0)
Maryland 2.74 4-8 Lost to Penn State 66-3 (0)
Illinois 1.2 2-10 Lost to Northwestern 42-7 (8.4)

There is plenty to quibble with about these satisfaction predictions. Looking at the final game satisfaction figures, for example, it seems to my eye that they are probably too low for teams that won a bowl game. For most fans, winning a bowl game is likely more satisfying than winning a regular season game for any given margin of victory and performance against the Vegas spread. And in particular I think the model clearly undervalues the impact of beating a highly ranked opponent in a bowl game, even in these cases where the B1G team was favored. As a result of this, those teams’ final season satisfaction ratings should probably be higher than they are predicted here.

The reason the model misses on this is simple: so far we have no Michigan bowl victories and zero victories over ranked opponents in our satisfaction database. Until we do we’re stuck guessing at how much those things affect the predictions. Likewise, since we only have one season’s worth of data we can’t model the effects of teams significantly outperforming (or underperforming) season expectations. Going 7-6 is worse than 8-5, but Boilermaker fans are looking at their 7 wins through a very different lens than Michigan fans are viewing 8 wins. Similarly, OSU is close to the top, but how satisfied can the Bucks really be at this point with a two-loss season? And what about Wisconsin? Was that a great season or was that like winning a silver medal and wishing you’d won the damn gold?

Looking at the results from 30,000 feet, however, they make sense. Thanks to the fact that game satisfaction is a big driver of how fans rate the season, the seven teams that won their bowl games generated higher season satisfaction scores than Michigan. It’s important to remember here that this is an analysis of fan satisfaction – the fact that the satisfaction rankings don’t mirror objective measures of season quality (i.e. win/loss records) is pretty much the whole point. Fans are emotional, irrational, and short-term thinking animals. We have the S&P to tell us how good teams are. We have the satisfaction index to have fans tell us how they feel about the teams.

For our grand finale, in case you want to compare Michigan’s roller coaster of satisfaction with others on a week-by-week basis, I leave you with the season trends for each of the B1G teams.

Michigan State (10-3)

Ohio State (12-2)

Wisconsin (13-1)

Penn State (11-2)

Purdue (7-6)

Northwestern (10-3)

Iowa (8-5)

Indiana (5-7)

Michigan (8-5)

Minnesota (5-7)

Rutgers (4-8)

Nebraska (4-8)

Maryland (4-8)

Illinois (2-10)

Fan Satisfaction Index: Week 6 Results

Fan Satisfaction Index: Week 6 Results

Submitted by OneFootIn on October 11th, 2017 at 9:51 AM

Well, that happened. Pretty much all the ingredients for a massive dip in fan satisfaction were present on Saturday:

  • Heartbreaking loss to a rival who had no business being any good this year? Check.
  • Michigan favored by double-digits? Check.
  • At home, under the lights, on national television? Check.
  • Horrendous, nigh-unwatchable performance by the offense? Check.
  • Highly questionable coaching decisions and play calling? Check.
  • Mind-numbing displays of God’s anger at Michigan in the form of turnovers and bad luck? Check.
  • Sinking feeling that this portends terrible things for the future? Check.

The loss came on top of rumblings and concerns about whether this team is really ready for prime time. Had the team taken a step forward during the bye week and paddled Sparty, the narrative would have shifted decisively for the positive. Instead, the disappointing loss did just the opposite, igniting complaints about Harbaugh and likely signaling an end to the honeymoon phase of Harbaugh’s tenure in Ann Arbor. It’s amazing how quickly fans have moved the goalposts. When Harbaugh came to town no one imagined Michigan would win 10 games in his first year. This week, the same folks ready to canonize Harbaugh are questioning his coaching ability.

Figures 1 and 2 provide our first look at how the fan base feels after a loss. It ain’t pretty. According to our respondents, the loss not only felt bad in and of itself, but it also pretty much destroyed the satisfaction fans were feeling about the season up to this point. The average game satisfaction plummeted to 20.8 – down from 76.2 after the Purdue game. The season to date score averaged just a 40.5 – also down from about 76 last week.

Figure 1: MSU Game Feelz 

 

 

Figure 2: Post-State Season Feelz

The loss also appears to have undermined confidence in the future. Asked to assess Michigan’s win probability next week against Indiana, Michigan fans averaged just 66.3%, considerably lower than the 78.4% S&P had projected during the bye week (but more in line with the new Vegas spread, which is now just 5 points).

Sadly, given how bad Michigan’s offense looked, one has to assume that the 66.3% is only that high given the confidence fans still have in the defense, which acquitted itself extremely well once again. It will be interesting to see how much fan satisfaction manages to rebound when Michigan wins next week. I don’t think any of us want to see what happens if Michigan finds a way to lose to Indiana. I’m not sure numbers even go there.

 

Themes, Thoughts, and Trends

Recency Bias and Fan Satisfaction

One look at fan evaluations of the season after the State loss provides a powerful suggestion about the importance of the last game played in the fan’s mind. Fans, more than most people, seem susceptible to the “what have you done for me lately” disease. Of course, it’s also possible that rivalry games are just much more important than other games and that losing to State ruined any good vibes folks were having to this point.

So I went back to the bye week data. I ran a regression model using people’s ratings of season satisfaction so far as the dependent variable and their ratings of the first four games as the independent variables. In English, that means the result of the regression tells us how much influence each individual game had on people’s evaluation of the season.

The results were fascinating. The adjusted R2 was .49, meaning that we can explain roughly half of the variation in people’s assessments of the season by knowing their assessments of the individual games. Not bad. But the kicker was that only one variable turned out to be statistically significant: the Purdue game.

I’m not sure that says much for the mental acuity or mental health of the average fan, but it is evidence for the recency effect. My tentative conclusion at this point is that rivalry games have an outsized impact on fan assessments, but so does the last game played. And right after a rivalry game, those two factors combine to produce a massive effect on fans (And it probably explains why Brian and Ace complain so much about their Twitter feeds after games like this one).

 

The Season So Far

We riding the roller coaster now, folks. As usual, the columns represent game satisfaction; the line shows season satisfaction ratings.

Figure 3: The Season So Far, Summarized