Fan Satisfaction Index: Week 4 Results

Submitted by OneFootIn on September 27th, 2017 at 9:44 AM

That’s better. At least, according to 309 of you, the fans, it was better by far than the last couple weeks. The victory over Purdue led to a big leap in fan satisfaction, from 62.9 last week to 76.2 this week. There was also a lot less disagreement about how to rate this game – the standard deviation this week was just 8.7, compared to about 15 last week.

Figure 1. Post-Purdue Satisfaction

Is this an obvious result?

On the one hand, the vectors promoting increased fan satisfaction included (at least) the following. Michigan:

  • Won its first true road game of the year
  • Started off Big Ten play with a solid win, beating a chippy, trash talking, and ascending team that several people had picked to upset the Wolverines
  • Welcomed the Big Ten’s next hot shot young coach with a beat down
  • Got revenge for an egregious (and uncalled) targeting penalty that knocked out our starting quarterback
  • Dazzled the nation with its defense again
  • Saw its tight ends emerge as the real threats we knew they could be
  • And holy crap John O’Korn!!!

On the other hand, there were at least two very large negatives that should be putting a damper on fan satisfaction today. Most obviously, Michigan suffered the potentially devastating loss of its starting quarterback and Speight’s timeline for returning remains a mystery. Yes, many folks were only too happy to see O’Korn take the wheel, but it is a bit curious that satisfaction could be so high in a game where your starting QB got injured. I’m trying to remember a game where folks came out feeling so sanguine after losing such an important leader – on either side of the ball. Can’t do it just yet.

The other troubling note was the horrendous play of the offensive line. Michigan’s sackapalooza lost a little glory thanks to the fact it gave up 4 sacks and 8 TFLs to Purdue’s sub-par defense. The running game sputtered as a result, too. Taking out Chris Evan’s 49-yard touchdown run, Michigan managed just 90 yards on 43 carries. Woof.

So why did this week’s game get a 76.2? Would it have been higher if Speight had stayed in the game and Michigan and eked out a narrower victory? I doubt it. It seems to me that John O’Korn’s amazing second half performance was dazzling enough to overcome the more sobering aspects of the game. But did Speight’s injury and the poor offensive line play keep the game rating from being even higher? Maybe a little. Who knows?

That’s the beauty of sports and being a fan – you can feel however you want, for whatever reason strikes you. That’s why we do the surveys. You can hypothesize all you want but never know exactly what you’re going to get until to take the poll.

Figure 2. Week 4 Season Satisfaction

When it came to updated feelings about the season, however, the Purdue bump was not nearly so big. This week season satisfaction was at 76.7 compared to 72.7 last week, but did not quite make it back to the 77.2 recorded after the Cincinnati game. This makes sense: as Michigan plays more games not all of them will produce enough new information to move the needle. And it seems likely that the offense’s continued struggles and Speight’s injury weighed more heavily in people’s assessments at the season level. Michigan is undefeated, sure, but many of us worry that Michigan’s offense won’t improve enough in time to knock off Penn State, Wisconsin, and Ohio State, despite the play of the defense.

Themes, Thoughts, Trends

They Saw a Game: The Social Construction of “Takes”

Over the past two weeks we’ve confirmed what everyone pretty much knew: hot takes are all over the map compared to cold takes. As the emotions of the moment fade, we’re less likely to overreact to our most recent experiences, good or bad, and judge things a bit more rationally.

But what also happens is that people talk to each other, trying to interpret what they saw. At first, people might disagree violently about the game – Speight was fine/Speight sucked/etc. Eventually, after enough debate and deliberation, the crowd usually finds a consensus narrative, or a pair of competing narratives.

This week’s hot take/cold take figure provides visually compelling evidence for this process. The average game satisfaction didn’t change too much from Sunday to Tuesday, but as the figure shows, the dispersion of takes certainly did. And as the bonus table indicates, Sunday’s hot takes were lower and higher than those on Monday, which in turn were higher and lower than those on Tuesday. The standard deviation of ratings on Tuesday was just half of what it was on Sunday. In English, by Tuesday the crowd had really started to coalesce around the game narrative.

Figure 3. Hot, Cold, Colder

Bonus Table: Purdue Game Satisfaction by Day

Day Mean Min Max Std Dev
Sunday 75.8 40 100 10.2
Monday 77 51 90 7.7
Tuesday 76.7 65 88 5.5

Measuring Momentum: The Season So Far

We still only have three games worth of data (repeat after me: “small sample size!”). But now that we have a few weeks of data we can start making comparisons anyway, generating hypotheses to test as we move on down the road.

The next two charts are “KDE plots” – basically smoothed out histograms. You lose the easy-to-read units on the Y-axis but you can compare the shapes of the distributions more readily and this format makes it easier to plot them together.

One big question in my mind for the future is how much the epic destruction of Sparty in a night game will move the needle. 

Figure 4. Game Satisfaction Comps: Cincinnati, Air Force, Purdue

Figure 5. Season Satisfaction Comps: Weeks 2-4

Figure 6. The Season So Far


The Maizer

September 27th, 2017 at 10:43 AM ^

Are you going to do one for the bye week? Would be interesting to see if it differed from the week 4 results.

Nitpicky request: Fig. 4 and Fig. 5 should have the same axis ranges.


September 27th, 2017 at 1:13 PM ^

Thanks for the note - I'm using the Seaborn package for Python to create the figures and my expertise does not yet extend to making the axes start at 0. The default is that the axis ends at the lowest value. It's on my list.

Re bye week survey: Yes, I do. I figure it's a good time to ask some different questions and I have some other musings to share based on Vegas point spreads, Brian's game previews, etc.



September 27th, 2017 at 11:21 AM ^

but...realistically ...after 4 games....we are good not great and the offense which at some point would be called on to win a game isnt up to the task..I doubt our defense as good as it can dominate a PSU, Wisc or OSU...with 2 of those on the road...we finish regular season 10-2


September 27th, 2017 at 1:21 PM ^

I think the Hot Take Dissipation Decay theory is a reasonable hypothesis, but you could just be seeing a low sample size thing... I bet you get fewer and fewer responses as time goes by which means you're less likely to see data at the tails of the disribution (i.e fewer Hot/Cold Takes).

Luckily, there's a way to control for this statistically: it's called the Test for Equality of Two Variances (aka: F-Test). The Excel Data Analysis Tool Pak Add In includes this analysis (see link below). If the P-value you end up with is less than the alpha value you specify (usually 0.05 for 95% confidence), then you can reject the null hypothesis and accept the alternate hypothesis. In this case, the null hypothesis is that the variances are equal and that there is no such thing as Hot Take Dissipation. The alternate hypothesis is that the theory is real.

Beware that failure to reject the null does not mean disproval of the alternate, merely that there is insufficient evidence to make the claim. Basically you may just need more data. The smaller the difference you're trying to detect, the more data you need to reject the null. 

Cool diary project!


September 27th, 2017 at 3:08 PM ^

Yes, I noticed that about the standard deviations, but the coclusion being drawn should still be validated. The reason the standard deviation recedes could be -- and likely is -- because the spread in each day is smaller due to the fact that with fewer samples you are less likely to see extreme data points, hence the diminshing standard deviations... In order to actually conclude "yes, these variances are indeed different" you should run the F-test. I'm not making this up, I swear. This is the very reason the test was invented by people smarter than us.

It's an analytical diary series, the data is available, and the anlysis is simple... so,what's the problem?


September 28th, 2017 at 9:19 AM ^


Thanks for the comment. I know lots of folks here love the details, but I am trying not to overwhelm people with statistics. I use them in my day job and can appreciate that there is a limit to how much people want to absorb when they're reading a blog about football.

But! Since you wondered - the p-value on the differences between the standard deviations is about .01 for each case. Not that surprising since we actually have a reasonable sample size (131 Sunday, 147 Monday, 30 Tuesday).

Thanks for reading.


September 27th, 2017 at 3:09 PM ^

I'm not trying to overthink any of this. There are clearly outliers each day. Sometimes its best to not think too much about the fancy test and trust what your eyes are telling you. Its right there, and its quite clear.

EDIT: meant as a reply to MCalibur


September 27th, 2017 at 4:33 PM ^

I see. Why think when we can just beleive whatever we want, is that it? 
The interpretation of the data in the article is incomplete and I offered a simple and readily available tool to improve the analysis. My hope is that the OP doesn't share your wanton ignorance and flippant wrongness.
Enjoy bliss.

The Maizer

September 27th, 2017 at 4:48 PM ^

Agree completely here. The whole point is an attempt to quantify something subjective. Not overthinking it and "trusting your eyes" is the opposite of that.

It would be akin to saying "don't look at the results of the survey, just look at the comments in the post about the survey and trust your eyes about how satisfied the fan base is."


September 28th, 2017 at 8:11 PM ^

And no x-ray machines within the entire State of Indiana, do we see O'Korn leading the offense down the field through the red zone for score? I dislike witnessing a kid injured but damn if we don't have new spark in our offense. Which most here have been begging for. Let's see what O'Korn can do on the field against Staee and Speight can heal at least unless and until O'Korn should fail to move the O downfield. Our "satisfaction" is high with that droid.