Sunday, January 11, 2009

Are Lottery's Really Fair?

The December 2008 Issue of Trailrunner Magazine included an interesting article on the growth of our sport, Ultrarunning titled Growing Pains. In the article, author Garrett Graubins discusses the lottery systems that many races are being forced to engage themselves in. “And while many races use online registration or first-come first-served mail-in entries, others have resorted to lottery systems in an attempt to give runners a fairly weighted chance of entry. In some lotteries, every entrant is given an equal chance of being picked. In others, entries are weighted according to a number of factors..”

Weighted chance of entry. This means that you have a chance of getting in… but it’s not going to be good. For those of you who were waiting to get into Western States as a Two Time Loser, you know your feelings on the weighted lottery system are pretty strong. I was upset too knowing that I had qualified for the race, but was going to be missing out on climbing Squaw in June again and again because I only have an 11% chance of getting picked in the lottery.

“In some lotteries, every entrant is given an equal chance of being picked.” Are they really though? Are we really being given an equal chance to be picked to run in their race. I’ve scoured the internet for races with lottery systems in place and want to focus on two of the tougher ones. I’m going to leave out Western States because it’s been talked to to death and feelings are a bit over the top in the community. (or are they.) Instead, I’m going to focus on Massanutten (MMT) and Hardrock. As an aside, I’ve all ready run at MMT and was a lucky winner in the lottery for 2009, so I will be returning.

Massanutten Mountain Trails 100
The system is a two-step process. You fill out the application, they have a lottery and, if you "win," you go back to the Web site to pay the entry fee. If you "lose" the lottery, you can get on the waiting list.
Priorities: The initial field will be selected randomly. Each qualified runner has an equal chance of being in the initial starting field. We will, however, have priorities on the waiting list. Given our experience with movement off the waiting list, we expect that these priorities will be significant.
Here are the priorities. Within groups, ranking is by the random number that is assigned in the entry process.
1. Prior overall winners of the men's or women's division of the MMT
2. Applicants who, on three prior occasions, served as pre-committed volunteers at the MMT
3. Finishers of five or more MMTs, ranked by number of finishes
4. "Two time losers" -- applicants who did not get in last year's MMT after being in the lottery, joining the waiting list, and staying on the waiting list until the end and then lost this year's lottery. (Here is the list of those who, if they lose the lottery this year, will qualify as two time losers.) All other applicants will be ordered by number of finishes (4 down to none).

This process seems pretty fair. Everyone is initially given an equal chance to be entered into the race. It is the WAITING list that is weighted. So unlike Western States, where the lottery itself is weighted, MMT only weights the waiting list. Your chances of getting into the race initially is 1 in how ever many enter the lottery. But if you’ve never run MMT and get placed on the waiting list, your chances of getting off the waiting list depends on how many repeat runners are trying to get in. The chances of getting off of the MMT waiting list have always been good, but with the new lottery in place, we could see a chock full waiting list as the years progress making your chances of getting off slim to none.

I spoke to Race Director Stan Duobinis about the MMT system and here is what he had to say:

SJ: Stan, How did the MMT committee come up with the current lottery system?
SD: The MMT lottery system was a compromise. At first, our Board of Directors wanted to have special categories that would get priority in the initial lottery. I did not want any preferences. I wanted a truly fair lottery. I use the word “fair” as it would be used by a statistician -- no weighting, no favoritism, every entrant has an equal chance of being chosen. (I'm an Econometrician -- if you don’t know what that is, it is essentially an economist who is also a statistician.) At the same time, I thought that we should reward our volunteers and possibly allow for repeat winners. So, we compromised, and created a fair lottery, but put special preferences in place for the waiting list.

SJ: Do you think weighting the wait list is fair?
SD: The use of the word fair implies that the system is an arbitrary one. It is arbitrary! I've had people tell me that this factor or that factor should count for setting priorities on the waiting list. Many of them choose a factor that might move them up on the waiting list.

When you get to trying to define the word fair, it becomes whatever you or I or some group decides. Your are suddenly in the normative world -- judgment. This means that you can argue about what you have done, but there is no right or wrong. What SHOULD be done is based on HOW YOU FEEL.

SJ: How do you feel about the lottery systems in General, necessary, nuisance, other??
SD: We are the victims of our own success. I and many others have helped to make ultras popular. Most of my "work" has been to simply talk to people about how great the events are and what how good it feels to finish one. Unfortunately, the most popular ultras appear to be ones that involve a challenge, an adventure and some exploring. Trail ultras have grown in popularity much more than road events. In fact, many road events have disappeared. That is fine except for one factor. While there are many trails at MMT and plenty of space to run so that we could easily accommodate double or triple the entrants on the trails, the parking at the various aid station locations is such that crews are now overwhelming those facilities. In fact, for MMT, the limiting factor on entry is really the crews and the cars. National forests don’t have lots of parking lots, and that's a good thing!

So, given the increased popularity how do we deal with 350 people wanting the 175 spaces at MMT this year? We hold a fair lottery. By the way, I have found that everyone who bothered to comment on our system of entry found a fair lottery to be a good system.

The Hardrock 100

Entry: This is a "post graduate" run. For safety reasons, we cannot accept entries from minors and novice runners. To apply to participate in the 2009 HRH you must satisfy at least one of the following qualifying standards.

i) finished any Hardrock within the past 5 runnings;
ii) started Hardrock within the past 3 runnings;
iii) completed any one of the following during 2007, 2008: Wasatch, Eagle, Bear, Leadville, Angeles Crest, Massanutten, Western States, Plain, HURT, Bighorn, Grand Teton, Tahoe Rim Trail, Tour de Mont Blanc (163 km version), or Cascade Crest Classic; or completed HURT in 2009.
iv) demonstrated equivalent mountaineering experience;
AND, you must be approved by the Runner Selection Committee.

Candidates not living in North America (who may not have access to the runs listed above) and any other candidates who have not run one of the qualifying 100 mile runs but wish to be considered under standard iv should submit a 500 word (or less)
account of their ultra-running/mountain experience that convinces the selection committee that they are prepared for the run.

North Americans must understand that to qualify under standard iv requires demonstrated exceptional mountaineering AND trail running experience. The Board of Directors rarely allows North American applicants into the event under
standard iv.

Entries postmarked no earlier than January 1, 2009 and received by selection day will be considered. The Board of Directors will select entrants on or about February 1, 2009 and the selected runners will be notified immediately thereafter. A wait list will be maintained after the field is full. If you are on the wait list and do not receive a spot, your check will be destroyed after run weekend.

The hardest part of getting into Hardrock is qualifying. The standards are set high, you must have completed one of the toughest 100 milers in the country to even be considered for their lottery. If you are selected, you are guaranteed a qualifying spot in the lottery in any one of the following 3 years. In my opinion, this lottery seems the fair given that there is no two-time loser rule, which proved disastrous for the Western States contingent, and nothing is weighted., not even the waiting list. The only thing that can save you here is your own ability to qualify, get lucky and show up on the starting line in July. But then again, what I think is unfair, is the process of giving extra tickets to folks who qualify under multiple standards. This, in essence, keeps many new comers out of the race and most of the new timers become repeat runners. This is exactly the type of system that is going to continue to quell the steady growth of our sport and I think old school ultra-runners and RD’s know it.

I asked one of the Board members of the Hardrock 100, Blake Wood Acting President of the Board of Directors, a few questions and here is how it went:

SJ: How did the Hardrock committee come up with the current lottery system? What do you think is the fairest part of how the system works?
BW: I'll answer these together. About five or six years ago when we realized that we were going to get more applications than our permit limit, the Hardrock Board met to discuss how we were going to choose who would get to run. There were a variety of options considered - some only half-serious. Among these were a lottery, a policy of first-come-first-served, a policy of simply choosing who we were going to let in (i.e. choose our friends), and (the half-serious one) choosing who to let in depending on how large an entry fee they sent. We decided a lottery would be most fair.

The next question was "what kind of lottery?" There were many runners who had stuck with us back when our numbers WEREN'T growing, and when it wasn't clear that Hardrock was going to be a viable run. We felt that we should reward them for their loyalty. We also wanted to reward those who have contributed to putting on the run. At the same time, we didn't want to shut out newcomers. We decided that a weighted lottery would be the best way to balance these goals.

Charlie Thorn and I came up with the basic algorithm - runners with more than a certain number of finishes are grandfathered in (to reward loyalty), extra tickets were given based on the number of DNFs and DNSs a person has (to reward loyalty and to help those who have sat on the wait list), extra tickets were given for work on the Hardrock trail and substantial service to the run (to encourage contribution to the run), and extra tickets were given to top-5 finishers from the previous year (a compromise between our policy of NOT emphasizing the competitive aspect of the run and the general community perception that the best runners ought to be encouraged to enter.) This algorithm was in effect for four or five years.

This past spring we decided to revise our lottery algorithm to remedy some perceived shortcomings, in particular, that it was too hard for newcomers to get in, that we didn't give enough extra consideration to those who had sat on the wait list, that we unduly rewarded some runners who rarely finish, and that we didn't sufficiently reward winners of the run. The revisions included changing how we determine who is grandfathered in, no longer giving extra tickets for DNFs, and automatically accepting the previous year's winners. This also had the effect of increasing the chances for newcomers to get in - basically because there were fewer total tickets in the lottery. Our acceptance rate for first-time applicants is somewhere around 25%, whereas it would be around 40% for a straight one-person-one-ticket lottery (that is, last year we had 2.5x as many applicants as entry slots). These revisions were a compromise hammered out between members of the Hardrock Board of Directors.

SJ: What do you have to say to those who have never run your race, and are on your waiting list?
BW: I tell them what their chances are of getting in, and encourage them to improve those chances by continuing to apply and by coming to Silverton to participate in our trail work weekend.

SJ:How do you feel about the lottery systems in General, necessary, nuisance, other??
BW: Lotteries are like income taxes - a necessary evil. Like taxes, they are also an opportunity to encourage desired behavior - in our case volunteerism and loyalty to the run. Like taxes, lotteries work best when they are perceived as "fair", although that perception will certainly vary from person to person. Feedback we have received from unsuccessful applicants is overwhelmingly "I didn't like the result, but I can't think of a better system." A very few "elite" runners have blasted us for not letting them in automatically.

SJ: Do you think that awarding "extra" lottery tickets to some entrants is fair? Do you feel that this leaves many first timers out?
BW: Yes, I think it IS fair. It rewards those who have contributed to the run in some way, through hard work or long-time support. It provides first-timers a reasonable chance of getting in, and gives them a way to nearly double or triple their chance by doing trail work. It improves the chances for runners who have sat on the wait list in previous years.

SJ: It is sad that many of the better ultra’s in our country are having to resort to a lottery system but it does work better than a first-come first-served system of entry. First-come first-served only works for those with a quick trigger finger on the mouse, some down time to be able to sit and weight, plenty of money in the bank to throw away at a moments notice and a prayer. Some folks do not use computers or have no access to a computer during the times in which races open up their registration. Still, there are plenty of quality races in our sport that deserve the same numbers and respect as these high profile races such as Western States, Hardrock and MMT. Remember, there is NO SUCH THING as an easy 100.