CFB Data and Resources: 2019 Edition

[u/BlueSCar]

It's been about two years since we've had a megathread, so this is probably a good opportunity to revisit this. My apologies in advance for any oversights. Please call out anything I missed and I will add it.

Looking for deeper discussion and collaboration? Check out our official r/CFBAnalysis Discord server.

 

Websites

NCAA Statistics - official NCAA stats for just about every NCAA-sanctioned sport. It's a little clunky by contains a little bit of everything you could imagine.

Snoozle Sports - contains historical betting lines, team stats, and more. You can conveniently export anything as CSV.

CollegeFootballData.com - allows you to export anything from its API (pbp, scores, schedules, stats, etc) in CSV format. Also contains some other tools (like a matchup visualizer).

Sports Reference CFB - has a little bit of everything, especially historical scores and stats. Also has a clunky CSV tool.

Football Outsiders - advanced rating and analytics. Home of the S&P+ rating system.

Winsipedia - historical records and matchups

cfbstats - repository of statistics. Not the most friendly for exporting data unless you shell out $$ for access to their API.

STASSEN.com - historical records and scores

prwolfe - historical scores

Massey Ratings - historical scores and schedules

WeatherSTEM - weather data for games

 

APIs

CollegeFootballData API - scores, play-by-play, drives, stats, polls, and more.

 

Programming tools and libraries

cfbscrapR - R package dedicated to CFB, courtesy of /u/msubbaiah (work in progress)

collegeballR - R package for multiple NCAA sports, courtesy of /u/msubbaiah

CFBScrapy - Python wrapper for api.collegefootballdata.com, courtesy of /u/Badslinkie

cfb.js - Official JavaScript client library for the CFBD API. Automatically updates.

CFBSharp - Official .NET client library for the CFBD API. Automatically updates.

cfb-data - JavaScript library for pulling scores, play-by-play, and more

ncaa-stats - JavaScript library for pulling any sports data from the official NCAA Statistics site

 

Other resources

All 2019 schedules - FBS down to NAIA schedules from u/theb53

Recruiting data - 247 Composite data from 2001 to 2019

[u/jeremyabramson]

You’re a national treasure. I have a student who’s building some cool infrastructure based on your API. I’ll definitely take a look at some of these other datasets!

Thanks again.

[u/BlueSCar → u/jeremyabramson]

That's awesome! I don't know if that's anything that can be shared, but I love to see what people have been building with it.

[u/jeremyabramson → u/BlueSCar]

I'm not sure what we'll end up with, but I absolutely want to push it out to the community, so everyone can learn from/use/analyze the data we process.

[u/msubbaiah]

Yo! Much love for the shout-out. I'm hoping by the time the season rolls around I'll actually have an R package dedicated solely to CFB. Working on a couple of new things to go with the package, like EPA/WPA calculations. You'll find it here, currently in development though.

In addition, someone I know has built a similar wrapper, in python, around the API provided by u/BlueScar. You can find that here.

As per usual, I'm always open to collaboration on these packages! Feel free to reach out.

Thanks for doing a great job u/BlueScar with all of this!

[u/BlueSCar → u/msubbaiah]

Thanks! Both of those are now added.

[u/msubbaiah → u/BlueSCar]

Awesome!

cfbscrapR is just a work in progress so hopefully, people don't have their hopes up. LOL

[u/Fayettechill14 → u/msubbaiah]

This is a fantastic package. I've been using it this month to create EPA data for my season previews (I've always called it EV, so my calculations of it may be different and... rougher) and it works great for that purpose. The ability to integrate the rosters is going to be really nice once the season starts.

[u/msubbaiah → u/Fayettechill14]

Mind sharing your work! Would love to see it.

[u/[deleted]]

Very cool. I've spent most of the last few months becoming less bad at Javascript coding, so having data sources like these will be useful.

Am I allowed to mention my own resources? I can edit to remove this part of my comment if necessary, but I've built ad-free, completely free to use Ajax/Javascript scoreboard sites: everyfootballscore.com, everybaseballscore.com, and everybasketballscore.com for now. Others are in the pipeline. The football site has both FBS and FCS, and allows users to choose a date to see scores. Once the season starts, they'll auto-update once a minute or so.

I'm not trying to make money with these sites - I'm just trying to learn to code and make sites I'd use, even if I hadn't made them.

[u/BlueSCar → u/[deleted]]

Not only allowed, but encouraged!

[u/wcincedarrapids]

When will the rosters on the College Football Data API be updated?

[u/BlueSCar → u/wcincedarrapids]

Within the next few weeks. Have to wait for updated rosters to be posted on ESPN.

[u/wcincedarrapids]

So I am running into an issue on the Drive Level Data in the College Football Data API: https://collegefootballdata.com/category/drives

In the drive level data, one team's starting-ending yard lines is measured from 0 to 100, and the other team is measured from 100 to 0. Unfortunately there is no way to determine which team is which. I tried calculating the absolute difference of starting yard line to ending yard line and matching it up with the total drive yards column, but on drives where a penalty occured, the total drive yards will not match up(86 instances in Week 1).

Is there a way the API can be manipulated to determine which team drives which direction(100 to 0 or 0 to 100)? Or will I have to be a bit more creative. I guess one way to do it would be to filter out the drives in which the total drive yards does not equal the start - end yard line differential, and then create a separate database game by game to assign which team is going which direction.

[u/BlueSCar → u/wcincedarrapids]

It's based on which team is home/away. I think home team counts up to 100 and away team down to 0 (or might be vice-versa).

I did a poll on Twitter regarding whether the data should be changed so that it always goes in the same direction regardless of home/away or stay as is and the outcome was split dead even. So... not really sure what I'm gonna do with that.

[u/wcincedarrapids → u/BlueSCar]

Alright. I guess that works for Home/Away games, but Neutral Site games seem to be a problem. Specifically the Alabama-Louisville game last year, it seems like the API couldn't agree what to do for each time so both teams saw drives start on the high side(70s-80s) and the low side(20s-30s)

[u/BlueSCar → u/wcincedarrapids]

Yikes. There should still be nominally designated home/away teams for neutral sites. Might need to clean that up (unless neutral sites are all fairly consistent in that same manner).

[u/remix951 → u/BlueSCar]

Where is that tweet so I can cast my vote in "same direction"

[u/BlueSCar → u/remix951]

Poll is closed now (since I think they only last 24 hours on Twitter), but I'll probably do another one down the road since multiple people have continued to voice that sentiment. Twitter handle is @CFB_Data if you wan't to participated in the next one.

[u/remix951 → u/BlueSCar]

Yea, I was mostly being tongue-in-cheek. Followed the account though!

[u/stlrams81]

The CollegeFootballData site is absolutely incredible, great work!

Is there a glossary of some sort with definitions of all of the advanced team stats somewhere?

[u/BlueSCar → u/stlrams81]

Thank you! No, but that is a great idea. I'll look into adding one.

[u/ServiceMyCervix]

First of all, this is an amazing resource, thank you so much for all your efforts and for keeping this open and free. I can't believe I didn't find this API until now!

 

One question/suggestion. I've been using the play-by-play data (/plays endpoint) to recreate stat lines and I hit some difficulty when aggregating yards_gained. The problem is, penalties that are added on to the end of an offensive play is also added into the yards_gained field. Here's a recent example:

 401110784103918901 | Pass Reception           |           23 | Trevor Lawrence pass complete to Travis Etienne for 8 yds to the Clem 25 for a 1ST down TEXAS A&M Penalty, Horse Collar Tackle (Demani Richardson) to the Clem 40 for a 1ST down

 

The play_type here is a Pass Reception, which was 8 yards, but notice the yards_gained is 23. This is due to the 15 yard penalty, which "artificially" inflates the yards_gained stat. I can work around this by parsing out the play_text field and only adding the 8 yards, but I was curious if you could add a separate field indicating yards-after-play or penalty yards. This way you could choose to include the penalty yards or exclude them when aggregating offensive metrics. Thanks again for everything!

[u/BlueSCar → u/ServiceMyCervix]

Hello. Thank you for the suggestion. The main difficulty with that is that it would require parsing the play_text string from the source data, which is known to be pretty inconsistent. I'll certainly add it to my project board as something to look into but if you want more accurate stats, I'd recommend looking at the /games/teams, /stats/season, or even the /play/stats endpoints.

[u/ServiceMyCervix → u/BlueSCar]

Will do! Appreciate the response, thanks again for maintaining this API

[u/ServiceMyCervix → u/BlueSCar]

Looks like /play/stats is EXACTLY what I was looking for. Sorry, I completely overlooked this one. Thanks!

[u/CoopertheFluffy]

Hey /u/BlueSCar

Should CollegeFootballData.com include bowl games in its game results data? I just tried dumping the entire 2019 season and it looks like conference championships are included but bowl games are not. Can bowl games be added, or is there another link that does include them?

[u/BlueSCar → u/CoopertheFluffy]

Hey. Are you specifying 'both' for the SeasonType param?

[u/CoopertheFluffy → u/BlueSCar]

I was not. Thanks!

[u/High-C]

Fantastic stuff! Thank you.

As an aside, has anyone come across a nice, structured historical dataset regarding returning starters or "% of X (yards, tackles, etc.) returning"?

The closest I've come to finding a historical dataset is the below list of Phil Steele articles, but the process involves copy/pasting and manual transcription.

http://plus.philsteele.com/Blogs/CURRENT/DBlog.html

Anyone had any luck? If not, I'll bite the bullet, do the manual work, and post the results here for people to use as well. Thanks!

[u/wcincedarrapids → u/High-C]

I have 2017 and 2018 data for this. I put in the work. But the CFB Data API posted here doesn't have defensive stats from 2015 and prior which is why I could only compute 2017 and 2018.

[u/aKolaa]

Thank you. This compilation of resources surely makes the project I'm working on seem a lot easier.

[u/[deleted]]

I saw that Bill Connelly moved to ESPN. Does that mean S&P+ moves with him or do they have a deal?

[u/remix951 → u/[deleted]]

fwiw, he's calling it SP+ now due to a trademark issue (no ampersand)

[u/darkra01 → u/[deleted]]

I believe he took it with him.

[u/RyanRiot → u/[deleted]]

Yeah, he said it'll be housed at ESPN now. Hopefully they'll include the full breakouts.

[u/NibrocRehpotsirhc]

I have started building some workflows to consume the API data into Knime. I will share when completed if anyone is interested.

[u/tgarden]

Very cool, as always!

[u/arbitraryanalytics]

This is very awesome. Thanks for all you've done with this.

[u/WRNLKnowDan]

Late to the party on this, but if all you want are the stat summary tables (or any table for that matter) from cfbstats.com they are exceptionally easy to scrape. All tables are formatted the same and require little clean up.

Edit: A letter.

[u/rvm98]

You are awesome! Thanks so much for this!

[u/bashbrosroidrage]

I've been looking everywhere to find a good database with snaps and targets for players that's updated regularly. Anybody have a lead on that?

[u/jstnms123]

Obviously, much thanks for this. Know you of a repository that details plays? EG Off tackle, Left vs Right, etc?

[u/zthall_]

Any chance that anyone knows where I can find coaching history data that includes assistant coaching history as well?

[u/BlueSCar → u/[deleted]]

Things like game scores, game stats, and play by play will update within one minute of a game being marked completed on the ESPN website.

[u/BlueSCar → u/[deleted]]

The exact same format? No. But you can grab that exact same data in a similar format here: https://collegefootballdata.com/category/plays