Advanced Modeling Tools

Within this book, we have covered a wide range of models. For many people, these tools will be enough to advance your football analytics game. However, other people will want to go further and push the envelope. In this section, we describe some methods we use on a regular basis. Notice that many of these topics are interwoven. Hence, learning one topic might lead you to learn about another topic as well.

Bayesian Statistics and Hierarchical Models

Up to this point, our entire use of probability has been built on the assumption that events occur based on long-term occurrence, or frequency of events occurring. This type of probability is known as frequentist statistics. However, other views of probabilities exist.

Notably, a Bayesian perspective views the world in terms of degrees of belief or certainty. For example, a frequentist 95% confidence interval (CI) around a mean contains the mean 95% of the time, if you repeat your observations many, many, many times. Conversely, a Bayesian 95% credible interval (CrI) indicates a range for which you are 95% certain to contain the mean. It’s a subtle, but important, difference.

A Bayesian perspective begins with a prior understanding of the system, updates that understanding by using observed data, and then generates a posterior distribution. In practice, Bayesian methods offer three major advantages:

• They can fit more complicated models when other methods might not have enough data.

• They can include multiple sources of information more readily.

• A Bayesian view of statistics is what many people have, even if they do not know the name for it.

For example, consider picking which team will win. The prior information can either come from other data, your best guess, or any other source. If you are humble, you might think you will be right 50% of the time and would guess you would get two games right and two games wrong. If you are overconfident, you might think you will be right 80% of the time and get eight games right and two games wrong. If you are underconfident, you might think you will be right 20% of the time and get eight games wrong and two games right. This is your prior distribution.

For this example, a beta distribution gives you the probability distribution given the number of “successes” and “failures.” Graphically, this gives you Figure 9-1.

Figure 9-1. Prior distribution for predicting results of games

After observing 50 games, perhaps you were correct for 30 games and wrong for 20 games. A frequentist would say you are correct 60% of the time (30/50). To a Bayesian, this is the observed likelihood. With a Beta distribution, this would be 60 successes and 40 failures. Figure 9-2 shows the likelihood probability.

Next, a Bayesian multiplies Figure 9-1 by Figure 9-2 to create the posterior Figure 9-3. All three guesses are close, but the prior distribution informs the posterior.

Figure 9-2. Likelihood distribution for predicting results of games

Figure 9-3. In this posterior distribution for predicting results of games, notice the influence of the prior distribution on the posterior

This simple example illustrates how Bayesian methods work for an easy problem. However, Bayesian models also allow much more complicated models, such as multi-level models, to be fit (for example, to examine a regression with both team-level and player-level features). Additionally, Bayesian models’ posterior distribution captures uncertainty not also present with other estimation methods. For those of you wanting to know more about thinking like a Bayesian or doing Bayesian statistics, here are some books we have found to be helpful:

• The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy by Sharon Bertsch McGrayne (Yale University Press, 2011) describes how people have used Bayesian statistics to make decisions through high-profile examples such as the US Navy searching for missing nuclear weapons and submarines.

• The Foundations of Statistics by Leonard J. Savage (Dover Press, 1972) provides an overview of how to think like a Bayesian, especially in the context of decision making such as betting or management.

• Doing Bayesian Data Analysis, 2nd edition, by John Kruschke (Elsevier, 2014) is also known as the puppy book because of its cover. This book provides a gentle introduction to Bayesian statistics.

• Bayesian Data Analysis, 3rd edition, by Andrew Gelman et al. (CRC Press, 2013). This book, often called BDA3 by Stan users, provides a rigorous and detailed coverage of Bayesian methods. Richard could not read this book until he had taken two years of advanced undergraduate and intro graduate-level math courses.

• Statistical Rethinking: A Bayesian Course with Examples in R and Stan, 2nd edition, by Richard McElreath (CRC Press, 2020). This book is between the puppy book and BDA3 in rigor and is an intermediate-level text for people wanting to learn Bayesian statistics.

Command Line Tools

Command lines allow you to use code to interact with computers. Command lines have several related names, that, although having specific technical definitions, are often used interchangeably. One is shell, because this is the outside, or “shell,” of the operating system that humans, such as yourself, touch. Another is terminal because this is the software that uses the input and output text (historically, terminal referred to the hardware you used). As a more modern definition, this can refer to the software as well; for example, Richard’s Linux computer calls his command-line application the terminal. Lastly, console refers to the physical terminal. The Ask Ubuntu site provides detailed discussion on this topic, as well as some pictorial examples—notably, this answer.

Although these command-line tools are old (for example, Unix development started in the late 1960s), people still use them because of their power. For example, deleting thousands of files would likely take many clicks with a mouse but only one line of code at the command line.

When starting out, command lines can be confusing, just like starting with Python or R. Likewise, using the command line is a fundamental skill, similar to running or coordination drills in football. The command line is also used with most of the advanced skills we list and will also enhance your understanding of languages such as R or Python. For example, understanding the command line will enhance your understanding of programming languages by making you think about file structures and how computer operating systems work. But which command line to use?

We suggest you consider two options. First, the Bourne Again Shell (shortened to bash, named after the Bourne shell that it supersedes, which was named after the shell’s creator, Stephen Bourne) traditionally has been the default shell on Linux and macOS. This shell is also now available on Windows and is often the default for cloud computers (such as AWS, Microsoft Azure, and GCP) and high-performance supercomputers. Most likely, you will start with the bash shell.

A second option is Microsoft PowerShell. Historically, this was only for Windows, but is now available for other operating systems as well. PowerShell would be the best choice to learn if you also do a lot of information technology work in a corporate setting. The tools in PowerShell would be able to help you automate parts of your job such as security updates and software installs.

If you have macOS or Linux, you already have a terminal with a bash or bash-clone terminal (macOS switched to using the Zsh shell language because of copyright issues, but Zsh and bash are interchangeable for many situations, including our basic examples). Simply open the terminal app on your computer and follow along. If you use Windows, we suggest downloading Git for Windows, which comes with a lightweight bash shell. For Windows users who discover the utility of bash, you may eventually want to move to using Windows Subsystem for Linux (WSL). This program gives you a powerful, complete version of Linux on your Windows computer.

(base) raerickson@pop-os:~$ pwd
/home/raerickson

raerickson@pop-os:~$  ls
 Desktop                 Games                 Public
 Documents               R                     Untitled.ipynb
 Downloads               miniconda3            Videos
 Firefox_wallpaper.png   Music                 Templates
 Pictures                test.py

cd ../

cd  /home/

cp test.py ./Documents

cp  ../test.py ./python_code

Version Control

When working on code on a regular basis, we face problems such as how do we keep track of changes? or how do we share code? The solution to this problem is version control software. Historically, several programs existed. These programs emerged from the need to collaborate and see what others did as well as to keep track of your own changes to code. Currently, Git is the major version control program; around the time of publication, it has a high market share, ranging from 70% to 90%.

Git emerged because Linus Torvalds faced that problem with the operating system he created, Linux. He needed a lightweight, efficient program to track changes from an army of volunteer programmers around the word. Existing programs used too much memory because they kept multiple versions of each file. Instead, he created a program that tracked only changes across files. He called this program Git.

Suggested Reading for Git

Many resources can help you learn about Git, depending on your learning style. Some resources we have found to be useful include the following:

• Git tutorials are offered on the Git project home page, including videos that provide an overview of the Git technology for people with no background.

• Software Carpentry offers Git tutorials.

• Training materials on GitHub provide resources. We do not provide a direct link because these change over time.

• Git books from O’Reilly such as Version Control with Git, 3rd edition, by Prem Kumar Ponuthorai and Jon Loeliger (2022) can provide resources all in one location.

Style Guides and Linting

When we write, we use different styles. A text to our partner that we’re at the store might be At store, see u. followed by the response k. plz buy milk. A report to our boss would be very different, and a report to an external client even more formal. Coding can also have different styles. Style guides exist to create consistent code. However, programmers are a creative and pragmatic bunch and have created tools to help themselves follow styles. Broadly, these tools are called linting.

Different standards exist for different languages. For Python, PEP 8 is probably the most common style guide, although other style guides exist. For R, the Tidyverse/Google style guides are probably the most common style.

To learn more about the styles, please visit the PEP 8 Style Guide, Tidyverse style home pages, or Google’s style guides for many languages.

Table 9-1 lists some example linting programs. We also encourage you to look at the documentation for your code editor. These often will include add-ons (or plug-ins) that allow you to lint your code as you write.

Packages

We often end up writing custom functions while programming in Python or R. We need ways to easily reuse these functions and share them with others. We do this by placing the functions in packages. For example, Richard has created Bayesian models used for fisheries analysis that use the Stan language as called through R. He has released these models as an R package, fishStan. The outputs from these models are then used in a fisheries model, which has been released as a Python package.

With a package, we keep all our functions in the same place. Not only does this allow for reuse, but it also allows us to fix one bug and not hunt down multiple versions of the same file. We can also include tests to make sure our functions work as expected, even after updating or changing functions. Thus, packages allow us to create reusable and easy-to-maintain code.

We can use packages to share code with others. Probably the most common way to release packages is on GitHub repos. Because of the low barrier to entry, anyone can release packages. Python also has multiple package managers including pip and conda-forge, where people can submit packages. Likewise, R currently has one major package manager (and historically, had more): the Comprehensive R Archive Network (CRAN). These repositories have different levels of quality standards prior to submission, and thus some gatekeeping occurs compared to a direct release on sites like GitHub.

Computer Environments

Imagine you run an R or Python package, but then the package does not run next session. Eventually, hours later, you figure out that a package was updated by its owner and now you need to update your code (yes, similar situations have occurred to us). One method to prevent this problem is to keep track of your computer’s environment. Likewise, computer users might have problems working with others. For example, Eric wrote this book on a Windows computer, whereas Richard used a Linux computer.

A computer’s environment is the computer’s collection of software and hardware. For example, you might be using a 2022 Dell XPS 13-inch laptop for your hardware. Your software environment might include your operating system (OS), such as Windows 11 release 22H2 (10.0.22621.1105), as well as the versions of R, Python, and their packages, such as R 4.1.3 with ggplot2 version 3.4.0. In general, most people are concerned about the programs for the computing environment. When an environment does not match across users (for example, Richard and Eric) or across time (for example, Eric’s computer in 2023 compared to 2021), programs will sometimes not run, such as the problem shown in Figure 9-4.

Tools like Conda let you lock down your computer’s environment and share the specific programs used. Tools like Docker go a step further and control not only the environment but also the operating systems. Both of these programs work best when the user understands the terminal.

Figure 9-4. Example of computer environments and how versions may vary across users and machines

Interactives and Report Tools to Share Data

Most people do not code. However, many people want access to data, and, hopefully, you want to share code. Tools for sharing data and models include interactive applications, or interactives. These allow people to interact with your code and results. For small projects, such as the ones some users may want to share after completing this book, programs like Posit’s Shiny or web-hosted Jupyter notebooks with widgets may meet your needs. People working in the data science industry, like Eric, will also use these tools to prototype models before handing the proof-of-concept tool over to a team of computer scientists to create a production-grade product.

Interactives work great for dynamic tools to see data. Other times, you may want or need to write reports. Markdown-based tools let you merge code, data, figures, and text all into one. For example, Eric writes reports to clients in R Markdown, Richard writes software documentation in Jupyter Notebook, Richard writes scientific papers in LaTeX, and this book was written in Quarto. If starting out, we suggest Quarto because the language expands upon R Markdown to also work with Python and other languages (R Markdown itself was created to be an easier-to-use version of LaTeX). Jupyter Notebook can also be helpful for reports and longer documents (for example, books have been written using Jupyter Notebook) but tend to work better for dynamic applications like interactives.

Artificial Intelligence Tools

Currently, tools exist that help people code using artificial intelligence (AI) or similar-type tools. For example, many code editors have autocompletion tools. These tools, at their core, are functionally AI. During the writing of this book, new AI tools have emerged that hold great potential to assist people coding. For example, ChatGPT can be used to generate code based on user input prompts. Likewise, programs such as GitHub Copilot help people code based on input prompts, and Google launched its own competing program, Codey.

However, AI tools are still new, and challenges exist with their use. For example, the tools will produce factual errors and well-documented biases. Besides well documented factual errors and biases, the programs consume user data. Although this helps create a better program through feedback, people can accidentally release data they did not intend to release. For example, Samsung staff accidentally released semiconductor software and proprietary data to ChatGPT. Likewise, the Copilot for Business Privacy Statement notes that “it collects data to provide the service, some of which is then saved for further analysis and product improvements.”

We predict that AI-based coding tools will greatly enhance coding but also require skilled operators. For example, spellcheckers and grammar checkers did not remove the need for editors. They simply reduced one part of editors’ jobs.

Conclusion

American football is the most popular sport in the US and one of the most popular sports in the world. Millions of fans travel countless miles to see their favorite teams every year, and more than a billion dollars in television deals are signed every time a contract is ready to be renewed. Football is a great vehicle for all sorts of involvement, whether that be entertainment, leisure, pride, or investment. Now, hopefully, it is also a vehicle for math.

Throughout this book, we’ve laid out the various ways in which a mathematically inclined person could better understand the game through statistical and computational tools learned in many undergraduate programs throughout the world. These very same approaches have helped us as an industry push the conversation forward into new terrain, where analytically driven approaches are creating new problems for us to solve, which will no doubt create additional problems for you, the reader of this book, to solve in the future.

The last decade of football analytics has seen us move the conversation toward the “signal and the noise” framework popularized by Nate Silver. For example, Eric and his former coworker George Chahrouri asked the question “if we want to predict the future, should we put more stock in how a quarterback plays under pressure or from a clean pocket?” in a PFF article.

We’ve also seen a dramatic shift in the value of players by position, which has largely been in line with the work of analytical firms like PFF, which help people construct rosters by valuing positions. Likewise, websites like https://rbsdm.com have allowed fans, analysts, and scribes to contextualize the game they love and/or cover using data.

On a similar note, the legalization of sports betting across much of the US has increased the need to be able to tease out the meaningful from the misleading. Even the NFL Draft, at one point an afterthought in the football calendar, has become a high-stakes poker game and, as such, has attracted the best minds in the game working on making player selection, and asset allocation, as efficient as possible.

The future is bright for football analytics as well. With the recent proliferation of player-tracking data, the insights from this book should serve as a jumping-off point in a field with an ever-growing set of problems that should make the game more enjoyable. After all, almost every analytical advancement in the game (more passing, more fourth-down attempts) has made the game more entertaining. We predict that trend will continue.

Furthermore, sports analytics in general, and football analytics specifically, has opened doors for many more people to participate in sports and be actively engaged than in previous generations. For example, Eric’s internship program, as of May of 2023, has sent four people into NFL front offices, with hopefully many more to come. By expanding who can participate, as well as add value to the game, football now has the opportunity to become a lot more compelling for future generations.

Hopefully, our book has increased your interest in football and football analytics. For someone who just wants to dabble in this up-and-coming field, perhaps it contains everything you need. For those who want to gain an edge in fantasy football or in your office pool, you can update our examples to get the current year’s data. For people seeking to dive deeper, the references in each chapter should provide a jumping-off point for future inquiry.

Lastly, here are some websites to look at for more football information:

• Eric’s current employer, SumerSports: https://sumersports.com/

• Eric’s former employer, PFF: https://www.pff.com

• A website focusing on advanced NFL stats, Football Outsiders: https://www.footballoutsiders.com

• Ben Baldwin’s page that links to many other great resources: https://rbsdm.com

Happy coding as you dive deeper into football data!