Rating Systems Explained

Teams in league sports often play balanced schedules. For example, in each season of the English Premier League, each football/soccer team plays each other team in the league exactly twice - once at their home stadium, and once at the opposing stadium. That way, at the end of the season, you can merely look at the record of wins, draws, and losses to determine which teams have done the best, second best, and so on. This works because everyone has had the same opposition, so wins against that same opposition are comparable.

But what would happen if different teams played different opponents, or even different numbers of matches? This is exactly the situation in many eSports, as well as chess; individual players may play different amounts of time against completely different opponents. To compare competitors in such as situation, we can use a rating system.

 

Reflection on an undergrad/grad crosslisted design of experiments course

 

I'm looking to re-examine and improve my teaching since I started working as a professor again for the University of Waterloo, and part of that means writing reflections on courses again. This one is for a 4^th year / graduate level crosslisted course on design and analysis of experiments.

Sampling, conditional probability, and random number generation

 Part of the motivation behind making the course Statistics and Gambling is to infuse new applicability into introductory or intermediate probability courses. This blog post is a look at how the course is going to cover familiar probability topics with examples in games of chance, and a simulation-based (rather than theory-based) approach.

This post covers basic methods of random number generation (RNG) in R, and applying RNG to demonstrate core concepts in sampling, conditional probability, and conditional distributions. It is meant to be a very surface-level primer on the topics, just enough to give context for the deeper dives into specific games of chance.

Wow, what are the odds? (Part 1: American Odds, Decimal Odds, and Implied Probability)

The term "odds" is slippery because it's used to mean different things in different contexts. In layperson terms, "odds" is often used as a synonym for probability. In proper statistical terms, "odds" is a function of probability, but it's not the same as probability. There are also other uses of the term "odds" in gambling contexts which are functions of a parallel concept called "implied probability". In these notes, we're going to look at some common types of odds in statistics and gambling contexts, and some of the calculations to convert between them.

 

How does Polychoric Correlation Work? (aka Ordinal-to-Ordinal correlation)

Let's say you've got data of many paired cases of two ordinal variables, like you might when you ask a large number of people the same two Likert scale questions (e.g. "poor", "fair", "good", "very good", "excellent").

 

What could you learn from the data from those two questions?

Here's a few common approaches:

 

1) Compare the means of each variable by abusing a t-test.

2) Compare the distribution of each variable with a chi-squared goodness-of-fit test.

3) Check for a relationship between responses of each variable with a chi-squared independence test.

4) Estimate the strength of such a relationship with a Spearman correlation.

Lottery tickets, baseball cards, and the coupon collector's problem

A man, a woman, an enby, and 27 elephants walk into a bar. The bartender looks at the group of 30 and asks "What is the probability that 2 or more of you have the same birthday?".

The group proceeds to disregard twins, leap years, and building codes. They talk amongst themselves, leave a little space for the reader to calculate or guess for themselves,

Fantasy Sports Explained

Information on fantasy sports ranges from the overly broad or impossible to apply like personal stories about how fantasy sports changed lives, to extremely narrow or short-lived like opinions on players recent performances. This article is intended to cover some of the middle ground to help you understand the basics of fantasy with a worked example of how you might choose players at the beginning of a league.

 

Textbook: Writing for Statistics and Data Science

If you are looking for my textbook

Writing for Statistics and Data Science

here it is for free in the Open Educational Resource Commons.

Writing for Statistics and Data Science is given out under the Creative Commons 3.0 - Attribution license. That means you and anyone else has the right to copy it, change it, even sell your version of it, as long as credit for the original continues to be attributed to me. In short, it's open source, have fun. 

There were a few chapters that I didn't include because they were either too niche or too prone to becoming obsolete. I'll be posting them here on the blog, with links being added in this post as those chapters go up. Details after the break.

 

Review of The Theory of Gambling and Statistical Logic

 

There are two reasons why I read Review of The Theory of Gambling and Statistical Logic, Second Edition (2009), by Richard A. Epstein, which dictated which of the text's 440 pages I paid attention to and which I skimmed.

 

First, to learn more of the fundamentals of betting strategy for my current job at Sportlogiq. Second, to get material to include in a possible future Statistics and Gambling course.

Statistics, Gambling, and Games of Chance

This is a proposal for a survey course on statistics that uses gambling extensively in examples. The target audience is senior undergraduates with a non-statistical background, but quantitative students will also find enough novelty to be interested. The goal of the course is not to encourage gambling, but to use it as a vehicle for a broad range of otherwise difficult statistical topics.

Basketball Data Science with Applications in R - An Advance Review

My overall impression of "Basketball Data Science" is that it's exactly the sort of book I would recommend to an instructor or able student of statistics in sport. Most of my criticisms are because it's not the book I imagine I would have written, not because the book isn't good.

Tips to successfully web scrape with a macro

Recently, I updated the cricket simulator that I made in grad school, which entailed gathering four years of new T20I, IPL, and ODI cricket data. That's about 1000 matches, and the website ESPNcricinfo has been dramatically updated since I first scraped it. For that matter, so have the tools in R for scraping.

However, for one part, the play-by-play commentaries, nothing was working, and I ended up relying on recording and repeating mouse-and-keyboard macros. It's crude, but the loading-as-scrolling mechanic was just too hard to deal with programmatically, even with otherwise very powerful Rselenium.

Using macros to scrape pages is a trial-and-error process, but with the following principles, you can drastically reduce the number of trials it takes to get it right.

Book review: Big Data by Timandra Harkness

I picked up Big Data by Timandra Harkness solely on the testimonials of Hannah Fry and Matt Parker on the front and back covers. The 2017 printing that I read is a 300-odd page general interest book about recent advances in big data.

"Big data" starts off pretty boilerplate for the topic – with a lot of definitions about what makes data "big"; volume, variety, velocity, and the like. It also gives some historical context about the growth of data over time, through early censuses, primitive computers, to today. The rest of this book is the result of interviews across the world with people working on different big data projects.

Replication report: Multilevel-Linear-Models using SAS and SPSS for repeated measures designs

The following is a report on the reproduction of the statistical work in the paper “Differences of Type I error rates for ANOVA and Multilevel-Linear-Models using SAS and SPSS for repeated measures designs" by Nicolas Haverkamp and André Beauducel at the University of Bonn.

Reading questions: Struck by Lightning

The book Struck by Lightning, by Jeffery Rosenthal hits that balance of scientific correctness and approachability just right for a general audience book on probability. It’s been in print for 14 years now, and was a Canadian bestseller, so there’s nothing new that would come from a traditional review and therefore I won’t write one.

Instead, two points:

1.      It should be required K-12 or 100-level math/stats reading.

Replication Report - Informative Priors and Bayesian Updating

The following is a report on the reproduction of the statistical work in the paper "The use of informative priors and Bayesian updating: implications for behavioral research" by Charlotte O. Brand et al.

The original paper was accepted for publication by Meta-Psychology, https://open.lnu.se/index.php/metapsychology, a journal focused on methodology and reproductions of existing work. This report continues my attempt to establishing a standard for replication reports according to the Psych Data Standards found here https://github.com/psych-ds/psych-DS

Package Spotlight: anim.plots

The package anim.plots behaves like a sort of user-friendly shell on top of animate that makes animations of some of the most common types of plots in base R in a more intuitive fashion that animate.

This package depends on two other important packages:

-   magick, which is an R implementation of imageMagick, which itself is software used to create animated gifs from still images.

-   animation, which is an R library that can be used to create animations from any collection of plots.

  

President's Trophy - A Curse by Design

There are lots of ways the NHL rewards failure and punishes excellence, like the player draft, ever shrinking salary caps, and the half-win award for participating in overtime, but even the way in which playoff pairings are decided has a perverse incentive.

There are three rewards to doing well in the NHL regular season:

-         1. Going to the playoffs,

-          2. A favorable first round pairing in said playoffs,

-          3. Home team advantage for playoff games more often.

Here I argue that the first-round pairings are not as favorable as they could be. 

Natural Language Processing in R: Edit Distance

These are the notes for the second lecture in the unit on text processing. Some useful ideas like exact string matching and the definitions of characters and strings are covered in the notes of Natural Language Processing in R: Strings and Regular Expressions

Edit distance, also called Levenshtein distance, is a measure of the number of primary edits that would need to be made to transform one string into another. The R function adist() is used to find the edit distance.

adist("exactly the same","exactly the same") # edit distance 0 
adist("exactly the same","totally different") # edit distance 14

Natural Language Processing in R: Strings and Regular Expressions.

In this post, I go through a lesson in natural language processing (NLP), in R. Specifically, it covers how strings operate in R, how regular expressions work in the stringr package by Hadley Wickham, and some exercises. Included with the exercises are a list of expected hang-ups, as well as an R function that can quickly check the solutions.

This lesson is designed for a 1.5-2 hour class for senior undergrads.

Contents:

• Strings in R
• Strings can be stored and manipulated in a vector
• Strings are not factors
• Escape sequences
• The str_match() function
• Regular expressions in R
• Period . means 'any'
• Vertical line | means 'or'
• +, *, and {} define repeats
• ^ and $ mean 'beginning with' and 'ending with'
• [] is a shortcut for 'or'
• hyphens in []
• example: building a regular expression for phone numbers 

• Exercises
• Detect e-mail addresses
• Detect a/an errors
• Detect Canadian postal codes