Featured post

Textbook: Writing for Statistics and Data Science

If you are looking for my textbook Writing for Statistics and Data Science here it is for free in the Open Educational Resource Commons. Wri...

Thursday, 16 February 2017

I read this: Chess Variants - Ancient, Regional and Modern

Chess Variants - Ancient, Regional, and Modern by John Gollon is a book published in 1968 that describes 33 different versions of chess, and includes sample games for most of them. It is the closest source I have yet encountered to answering the 'why chess' questions.

In the author's words, it is intended to be a Hoyle's Book of Games, but for chess. It also asrved as a humbling reminder to look outside university resources sometimes. The Bennett library may have nearly every book one could want on statistics, but it has perhaps 10 on chess, and those are mostly about artificial intelligence.

There's a lot of useful information from this book, much of the subtler implications are beyond me. Here's the gist of what I learned:

I read this: Chess Metaphors: Artificial Intelligence and the Human Mind

Chess Metaphors: Artificial Intelligence and the Human Mind, by Diego Rasskin-Gutman uses chess to explain concepts of intelligence both organic and artificial. The first part of the book is about the human mind, and cognitive models like schema and memory chunking, which I only skimmed. The second part discusses how an AI program can efficiently explore all the relevant consequences of any given chess move.

In the AI portion, 'Metaphors' explained how minimax algorithms worked, and discussed the Alpha-Beta algorithm, a chess playing staple, in particular. Minimax algorithms are useful for zero-sum two player games, like chess and most other head-to-head games.

Algorithms of this type work by compiling a set of possible moves, and for each move a set of possible responses by the opponent. The consequences of the response (e.g. the board state) is evaluated for each considered response. The worst (from the perspective of the AI) board state possible is assumed to be the response, and the evaluation (e.g. how 'good' that board state is) is recorded as the value of making that move. This repeats for each possible move, such that the AI has the worst case that comes from each possible move (each max). It then chooses the move that produces the best of the worst cases (the minimax). If someone playing against this AI doesn't respond with that optimal response, all the better.

The algorithm described above would only work for looking one move (one ply) ahead for each team. To consider deeper strategies, the process is repeated in an exponential explosion of possibilities. Reasonably, many of these algorithms differ in their focus on finding ways to avoid evaluating unnecessary positions, which is what Alpha-Beta does.

What really surprised me is how simple the evaluations can be. A very common evaluation method is to assign a value for each piece and a value for each square that piece can move to. This implies that such an AI would be functional, although not optimal, for a wide range of chess variants. In fact, if only the orthodox pieces are used, no additional programming would be required other than alter the possible moves to the new board. An algorithm like alpha-beta could be applied 'out of the box' to variants that don't use new pieces. This would explain why the 'chess variants' app that has the smaller boards like Garner's Minichess, and rearranged boards like Chess960 uses Alpha-Beta.

Including new pieces would involve programming in their possible moves and assigning them a material value (e.g. worth 4 pawns). Therefore, adapting existing AI to many of the variations seen in John Gollon's "Ancient, Regional, and Modern" book should be feasible. There are many ways to tune the relative value of pieces and immediate movement ability, including pitting AIs with different parameter values against each other in an evolutionary pool.


Footnote: Judging by http://www.chessvariants.com/ and the chess variants subreddit, it's much easier to make a variant than to drum up support and playerbase for it.

To consider later: Smess (also available as an app):
https://boardgamegeek.com/boardgame/1289/smess-ninnys-chess
In this game, the pieces have very simple moves and the board itself defines the difference in moves.


Saturday, 11 February 2017

Creating Homework Material for Statistical Writing Classes and Workshops.

This summer, I'll be offering a workshop on statistical writing to the graduate students in the SFU Stats department. In the year that follows, I hope to teach SFU's Stat 300 course, an undergraduate writing course that is mandatory for all statistics majors. I want this course to be offered at more universities, but since the typical undergrad degree in stats is already overloaded, this would need some additional motivation, so here's my anecdotal hook:

When graduate schools ask for a letter about your motivations for applying, or about your general background, there's a few things that are being examined. First, different graduate supervisors have different specialties, and they may be looking at your letter for signs of a good match. Second, the letter is a means of seeing firsthand your personal ability to communicate in English. They ask for something personal rather than a set topic to discourage plagiarism. They are not interested in your level of motivation; everyone says they are highly motivated.

I've recently been writing reference letters and filling out graduate school reference forms for students that have been in my previous 300-level classes. All of these schools are asking about the ability to write. Many statistics undergrads are applying to fields outside of pure statistics, for example economics, medical science, and business. All of them are asking their references about skills in English, often spoken English and always written. Likewise, the Statistical Society of Canada asks referees about spoken English as a part of their A.Stat accreditation.

In order to develop these skills in a program that usually focuses on mathematical theory and programming ability, I've sketched out some projects and exercises that could fit into a workshop or course.

The instructor (me) would ask colleagues from other fields that use quantitative data, such as sociology, anthropology, biology, business, ecological restoration, and history for reports from previous or current projects.

The students would read the reports and identify what the sampling method was, what analysis was done, and to check the assumptions on those methods to the data. They would interpret the results in their own words (not in the words of the discussion or conclusion section), and compose a couple of questions about the research that weren't answered. These questions don't have to pertain to the domain; they should pertain to the methodology. These would be questions like: were there any outliers that affected your results? Did you consider multiple testing? Did you consider regression/anova?

For example, for this research report, Water Quality of Stoney Creek and its Effects on Salmon Spawning, by SFU undergrads by Oak, Tony; Thai, Michelle; Orgil, Indra; Ngo, Kevin; Lu, Jerry , available at http://summit.sfu.ca/item/12770

A writing assignment could be:

  • What were the parameters to be estimated?
  • What biologically important thresholds or critical values are mentioned for the parameters being estimated?
  • What sampling method was used? How many sampling units are there?
  • Were there any null hypotheses being tested? If so, were they rejected? Should they have been?
  • Describe the qualitative differences between the four sites. Use the map and Figs 4,5,6, and 7.
  • In your own words, summarize the quantitative data described in Figures 2 and 3.
  • Compose two suggestions or questions about the analysis that may not have been considered.
  • Describe one way you could build upon (e.g. expand, follow up) this study, the type of information you would collect from this new work, and what additional conclusions you could make if the data matched your expectations.

(Note: Please don't take these questions as a criticism of this research report. It was selected to show how material is available within one's on campus. Even a quick skim of the report will show it's truly excellent work for undergrads.)


Another exercise I'm thinking of doing is to have students copy edit some scientific writing. This could be a publication or a blog post. I would do this twice; once with a well written work, and once in something of questionable quality like the unedited papers that come out of discount publishers. If I used something from ArXiv, with the authors' permission, I could use later, more complete versions of the same work as a 'key'. However using ArXiv is untenable for graded work because the 'key' is publicly available.


Also, there are the reading comprehension exercises.The ones I had posted earlier (here and here) were meant to be done in less than 2 hours, and had questions focused on specific statistical topics. Another possibility is to write more such assignments that require less statistical expertise, but more in-depth composition.

Tuesday, 31 January 2017

A Short Guide to Finding Brazilian Food in Canada.

(Um pouco sobre comida Brasileira no Canada.)



In the general spirit of welcoming people to Canada, I wanted to share something Gabriela, my significant other from Brazil, and I, have learned about Brazilian food in Canada. We've both heard frequent stories about Brazilians here missing the food, so I'm writing this in the hopes that it helps some people feel less homesick.



Bife, carne

Directly translated, 'carne' means 'meat', however in Brazil, 'carne' seems to refer only to 'red meat' such as cow meat. In Canada, 'meat' usually refers to any animal muscle tissue that is eaten, including fish, chicken, and pork.

The English word for cow meat is 'beef', which can refer not only to steaks (bife), but also to hamburgers and ground-up meat on pizzas.

The term 'filet-mingon' refers to a specific type of steak, one which is of very high quality and cost. Other steak cuts include 'New York' (Nova York), 'rib-eye', and 'porterhouse'. For a visual guide of the kind of meat you're looking for, refer to this guide ( http://culinaryarts.about.com/od/beefporkothermeats/ss/cutsofbeef.htm ) and find either a local or a supermarket butcher.


Bis

These little chocolates are unavailable in Canada, but Kit-Kat chocolate bars are quite similar.


Com gas / Sem gas

These terms refer to drinks with and without carbon dioxide gas bubbles added to them. The common Canadian terms for these are 'carbonated' and 'non-carbonated'. A less commonly used term for carbonated is 'sparkling'.


Feijões

Compared to beans in Brazil, canned beans in Canada are much sweeter, especially those with maple syrup. However, refried beans are essentially the same beans as those in Brazil – starchy and salty.


Globo

These crunchy, light, bagged snacks from Rio de Janeiro reminded me of Funyuns, but without the powdered onion flavour.


Molho Ingles (tipo Worcestershire)

This cooking sauce for meats is known as Worcestershire sauce in Canada. It is exactly the same as in Brazil.


Sonha de Valsa

Reese's peanut butter cups are a close substitute to these chocolates, but they taste less natural than Sonha de Valsa.


Pao de Queijo

These puffs of cheesebread are available (at least in Vancouver area) as a frozen food from a company called Otimo, as well as at least one competitor.


Pastels

Spring rolls are an imperfect substitute. These are pockets of flaky fried dough that are like small pastels. They can be found in several styles in different Asian restaurants. For example, Chinese spring rolls tend to be smaller and less meaty than Vietnamese spring rolls. At ethnic grocery stores, such as T&T Supermarket in Vancouver, you can find the sheets to make your pastels at home, and filled with whatever you want. So far, Gabriela and I have tried this with breaded chicken, cheese, crème de meil (creamed corn), and Sonha de Valsa.


Yakult

These tiny bottles of sweetened, probiotic yogurt drink are available as a different brand, called 'Wahaha' at T&T supermarket.



General Strategy – Read the French side

Most food packaging in Canada has both English and French written on it. Even without knowing French, reading the French side of the package can offer some more information about a food. If you know Portuguese (or Spanish), try to guess what the following French words mean, just by using Portuguese (or Spanish):

Sucre, framboise, menthe, limon, bifteck, riz, froid, chaud, crème, glace*.


Not food but…

You may have noticed that Canadian soccer is not as exciting to watch as it is in Brazil. However, I recommend asking who knows about (ice) hockey to show you a hockey game. From my personal experience, soccer fans tend to learn and enjoy hockey very quickly; it's a similar game, but with a smaller net and much more, and faster, substitutions.

Also, there's more to Canada than just Vancouver and Toronto. For example, the biggest shopping center in North America is in Edmonton, Alberta. The West Edmonton Mall has lots of other theme park style attractions too, so if you've been to Orlando, Florida and enjoyed the Aventura Mall and Disney World, I recommend it.


*Açúcar, framboesa, menta, limao, bife, arroz, frio, quente, creme, gelo **
**Sugar, raspberry, mint, lemon, steak, rice, cold, hot, cream, ice.

Saturday, 31 December 2016

Reflections on teaching Stat 342 1609

Stat 342 is a course about programming in the SAS language and environment. It is aimed at statistics major undergrads. This course was delivered as a single two-hour lecture each week for all 60-70 students together, and a one-hour lab with the students in smaller groups with a lab instructor.

This was only the second time the course has been offered at SFU, which introduced some challenges and opportunities that were new to me. It was also the first course I have delivered to an audience of statistics majors.
 
My biggest regret is not putting more content into the course, especially as assignments. I should have given four assignments instead of two, and allowed for much more depth and unsupervised coding practice. This is especially true with more open topics like the SQL and IML procedures, and data steps. SAS is an enormous topic, but I feel like I could have done more with the two credits this course entails.

My biggest triumph was the inclusion of SQL into the course. I covered what SQL was and its basic uses of inspecting, subsetting, and aggregating data. This meant a commitment of two weeks of the course that hadn't been included before, and wasn't in the text. I heard from two separate sources afterwards, as well as students, that learning SQL was a priority but it wasn't found elsewhere in the curriculum.

In short, my personal conviction that SQL should be taught to stats students was validated.


The textbook, titled "SAS and R - Data Management, Statistical Analysis, and Graphics." by Ken Kleinman and Nicholas J. Horton., was half of the prefect textbook.

Very little theory or explanation is given of any the programs in the textbook. It read more like the old Schaum's Outline reference books than a modern text. There were simply hundreds of tasks, arranged into chapters and sections, and example code to perform these tasks in R and in SAS. Since most of these students were already familiar with R, this meant they had an example that they already familiar with as a translation, in the short exposition wasn't enough. It was a superb reference; it was the first book I have declared required in any course I've taught.

Having said that, the book "SAS snd R" still left a lot of explanation for me to provide on my own or from elsewhere. It also lacked any practice problems to assign or use as a basis. I relied on excerpts from a book on database systems [1], a book on data step programming [2], as well as from several digital publications from the SAS Institute. You can find links to all of these on my course webpage at https://www.sfu.ca/~jackd/Stat342.html



SAS University Edition made this course a lot smoother than I was expecting. Installing the full version of SAS has a lot of technical difficulties due to legacy code and intellectual property rights.

Simon Fraser University has a license for some of its students, but it's still a 9 GB download, and it only works on certain versions of Windows. By comparison, SAS U Edition is 2 GB, and the actual processing happens in its own virtual machine, independent of the operating system on the computer. The virtual machine can be hosted by one's own computer or remotely through Amazon Web Services.

Actually using this version SAS just requires a web browser. I has a virtual machine set up on my home computer, and a copy running through Amazon. That way, I could try more computationally demanding tasks at home, and demonstrate everything else live in class from a projector. Also, students' installation issues were rare (and exclusively the fault of a recent patch from VMWare), but could be dealt with in the short term by giving access to my Amazon instance.


Exam questions were of one of three types; give the output of this code, write a program to these specifications, and describe what each line of this program does. Only the third type has room for interpretation. This made marking faster and the judgements clearer to students.


It's hard to make comparisons of the support burden of this course to others I have taught because it was much smaller. I taught two classes this term and the other was more than 5 times as large. Naturally, the other had more than 5 times as many questions and problems from individual students.

The nature of the tasks in the two assignments and on the two exams gave less opportunity for arguing for marks as well. The assignment questions had computer output that had clear indicators of correctness.


Compared to the audiences of 'service' courses (courses offered by the stats department in service to other departments), there are some differences that call for a change in style. Majors seem to be more stoic in class. That is, it's harder for me to tell how well the class is understanding the material by the reactions of the students. Often, there is no reaction. In some cases, I think I covered some ideas to the point of obviousness because i misjudged the audience (too many examples of the same procedure). At least once, I rushed through a part that the students didn't get at all (ANOVA theory). Also, my jokes never get a reaction in class. Ever.

On the flip side, these students seem more willing to give me written feedback, or verbal feedback outside of class. None of this should surprise me; as a student I have tried to blend into a class of three people.

[1] "Database Systems: Design, Implementation, and Management" by Carlos Coronel, Steven Morris, and Peter Rob.
[2] "Handbook of SAS DATA Step Programming" by Arthur Li.

Wednesday, 28 December 2016

Reflections / Postmortem on teaching Stat 305 1609


Stat 305, Introduction to Statistics for the Life Sciences, is an intermediate level service course, mainly for the health sciences. It is a close relative to Stat 302, which I had taught previously in its requirements, audience, and level of difficulty. Compared to Stat 302, Stat 305 spends less time on regression and analysis of variance, and more time on contingency tables and survival analysis.








 

Changes that worked from last time: 
Using the microphone. Even though the microphone was set to almost zero (I have a theatre voice), using it saved my voice enough to keep going through the semester. Drawing from other sources also worked. Not everything has to be written originally and specifically for a given lecture. Between 40 and 50 percent of my notes were reused from Stat 302. Also, many of the assignment questions were textbook questions with additional parts rather than made from scratch.


   
Changes that didn't work from last time: 
De-emphasizing assignments. Only 4% of the course grade was on assignments, and even that was 'only graded on completion'. This was originally because copying had gotten out of control when 20% of the grade was assignments. This didn't have the desired effect of given people a reason to actually do the assignments and learn rather than copy to protect their grades.
 


Changes I should have done but didn't:

Keeping ahead of the course. I did it for a few weeks, but it got away from me, and I spent much of the semester doing things at the last feasible minute. This includes giving out practice materials. On multiple occasions I watched f.lux turn my screen from red to blue, which it does to match the colour profile of my screen to the rising sun.




What surprised me:

The amount of per-student effort this course took. There were fewer typos than in previous classes, and therefore student questions about inconsistencies in the notes. However, there was an unusually large amount of grade change requests. Maybe there was a demographic difference I didn't notice before, like more pre-med students, or maybe the questions I gave on midterms were more open to interpretation, or both.




What I need to change:

My assignment structure. There should have been more assignments that were smaller, and ideally they should include practice questions not to be handed in. Having more questions available in total is good because finding relevant practice material is hard for me, let alone students. Having smaller and more assignments mitigates the spikes student workload, and means that the tutors at the stats workshop have to be aware of less of my material concurrently.




Tophat:

Tophat is a platform that lets instructors present slides and ask questions of an audience using laptops and mobile devices that students already have. My original plan was to use iClickers as a means to poll the audience, but Tophat's platform turned out to be a better alternative for almost the same cost. It also syncs the slides and other material I was presenting to these devices. My concerns about spectrum crunch (data issues from slides being sent to 200-400 devices) didn't seem to be a problem, but I



Scaling was my biggest concern for this course, given that there were more students in the class than in my last two elementary schools combined. I turned to Tophat as a means of gathering student responses from the masses and not just from the vocal few. It also provided a lot of the microbreaks that I like to put in every 10-15 minutes to reset the attention span clock.



However, Tophat isn't just a polling system that uses people's devices. It's also a store of lecture notes, grades, and a forum for students. This is problematic because the students already have a learning management system called Canvas that is used across all class. This means two sets of grades, two forums (fora? forae?), and two places to look for notes on top of emails and webpage.



To compound this, I was also trying to introduce a digital marking system called Crowdmark. That failed, partly because I wasn't prepared and partly because students' data would be stored in the United States, and that introduces a whole new layer of opt-in consent. Next term, Crowdmark will have Canadian storage and this won't be a problem.



I intend to use Tophat for my next two classes in the spring, and hopefully I can use it better in the future.





The sheep in the room:

During the first two midterms, there was blatant, out-of-control cheating. Invigilators (and even some students) reported seeing students copying from each other, writing past the allotted time, and consulting entire notebooks. There was no space to move people to the front for anything suspicious, and there was too much of it to properly identify and punish people with any sort of consistency. Students are protected, as they should be, from accusations of academic dishonesty by a process similar to that which handles criminal charges, so an argument that 'you punished me but not xxxx' for the same thing is a reasonable defense.
 

 The final exam was less bad, in part because of the space between them and attempts to preemptively separate groups of friends. Also, I had two bonus questions about practices that constitute cheating and the possible consequences. For all I know, these questions did nothing, but some of the students told me they appreciated them nonetheless. Others were determined to try and copy off of each other, and were moved to the front.
 

What else can be done? Even if I take the dozens of hours to meet with these students and go through the paperwork and arguing and tears to hand out zeros on exams, will it dissuade future cheaters? Will it improve the integrity of my courses? Will I be confronted with some retaliatory accusation?
 

Perhaps it's possible to create an environment where there are less obvious incentives to cheat. Excessive time pressure, for example, could push people to write past their time limit. Poor conditions are not an excuse for cheating, but if better conditions can reduce cheating, then my goal is met. But why a notebook? The students were allowed a double sided aid sheet; that should have been enough for everything.
 

This is something I don't yet have an answer for.
 


Priming:

The midterm exam was very difficult for people, and I anticipated a lot of exam anxiety on the final. On the final exam, I had two other bonus questions on the first page.



One of them asked the student to copy every word that was HIGHLIGHTED LIKE THIS, which was five key words that had been overlooked on many students' midterms in similar questions.



The other question employed priming, which is a method of evoking a certain mindset by having someone process information that covertly requires that mindset. The question was



What would you like to be the world's leading expert in?”



and was worth a bonus of 1% on the final for any non-blank answer. The point of the question was to have the students imagine themselves as being highly competent at something, anything, before doing a test that required acting competently. Most of them wrote 'statistics'. In literature on test taking, a similar question involving winning a Nobel Prize was found to have a positive effect on test scores in a randomized trial. It's impossible to tell if my question had any effect because it was given to everyone. However, several students told me after the exam that they enjoyed the bonus questions.



Priming is one of the exam conditions I want to test in a formal, randomly assigned experiment in the near future. It will need to pass the university's ethics board first, which it obviously will, but it's still required. It's funny how one can include something like this in an exam for everyone without ethical problems, but need approval if I want to test the effect because it's testing on human experiments. 

Facebook wound up in a similar situation where they ran into ethical trouble for manipulating people's emotions by adjusting post order, but the trouble came from doing it for the purpose of published research and not something strictly commercial like advertising.




Reading assignments:

 In the past, I have included reading assignments of relevant snippets of research papers using the methods being taught. Worrying about overwhelming the students, I had dropped this. However, I think I'll return to it, perhaps as a bonus assignment. There were a couple students that even told me after the class that the material was too simple, and hopefully some well-selected articles will satisfy them without scaring everyone else.

 

Using R in the classroom:

In the past, I also had students use R, often for the first time. I had mentioned in a previous reflection the need to test my own code more carefully before putting it in an assignment. Doing so was easily worth the effort.
 

Another improvement was to include the data as part of the code, rather than as separate csv files that had to be loaded in. Every assignment with R included code that defined each variable of each dataset as a vector and then combined the variables with the data.frame() function. The largest dataset I used had 6 columns and 100 rows; anything much larger would have to be pseudo-randomly generated. I received almost no questions about missing data or R errors; those that I did involved installing a package or the use of one dataset in two separate questions.


Saturday, 3 December 2016

2016 did not suck.

The idea 2016 sucked is an indication of the triumph of media over statistics.

In the United States, which I use because their data is very open and thorough, national unemployment is less than 5% and has been steadily dropping. Of those unemployed, most of them are either between jobs by choice, or have not yet ever had major employment. We know this because the proportion of the workforce on unemployment assistance is below 1.20%, the lowest rate in more than 40 years.
 

Also in the US there was also a record low number (not proportion, raw number) of teenage pregnancies. That means both fewer abortions AND fewer unwanted births. So if you're pro-choice or pro-life, your side is winning.

Globally, the birth rate is falling faster than previously forecast, largely because of faster-than-forecast improvements to the quality of life in India. Last month, the Indian government target for solar power capacity was raised dramatically foe the 3rd time in 6 years because the bringing clean energy to people keeps getting easier and cheaper at a rate faster than anyone could reasonably expect.

Compared to 2015, worldwide sales of electric cars has increased 55%, worldwide use of coal has decreased. Anthropogenic (Man-made) carbon emissions were the same as 2015, despite the world economy growing.  This is an indication that we could get climate change under control.


A lot of the problems of the last couple of years were either mitigated well in 2016, or outright handled.

Remember Ebola? The outbreak is officially over, and was close to over for most of 2016. There is a vaccine that's currently in use, and if something goes wrong, we have another vaccine candidate in Phase III (late human-equivalent testing) trials to take its place. This was the ultimate Ebola outbreak - not just the biggest but the last we will ever see.

Remember the Fukushima disa
ster? Radioactivity in most (not within a km of the damaged reactor) of the region has dropped to levels fit for permanent human habitation. After the disaster, Japan shut down all their nuclear power plants for years for safety upgrades, and 2016 saw the last of them come back online. What happened with the Fukushima reactor was extraordinary, and problem of that magnitude is impossible for CANDU modern reactors.

Remember Zika? The outbreak has been contained. There are scattered reports of new cases, but it's not showing up all over the world as was predicted after the Rio Olympics. On that note, the Rio Olympics seemed unremarkable as far as mishaps and problems are concerned. I saw Rio de Janerio in October. It looked like it has survived well enough.


Remember that war that started in 2016? Me neither. A cursory search finds no armed conflict between two or more different countries that started this year.

You can remember 2016 as the year we lost Alan Rickman, but 40 million people will remember it as the year they got access to clean water.