Hey there! It's been a while, but I'm back with another post about an interesting problem I've had recently. I'll quickly introduce the situation and then talk about how we solved it.
At SwampHacks, the University of Florida's flagship hackathon, organizers need to review and select attendees from a list of 1000+ applicants. Our team is relatively small, approximately 30-ish members who all work around different times. Within the organization, we also have smaller teams, such as the six-person tech team, who are more synchronized.
This makes it hard to agree on applicants to admit. We cannot feasibly coordinate everyone sitting down, reviewing applications at once, and making decisions together. Furthermore, each applicant may be on a team, and we would prefer to keep teams together in admissions. So how do we make this process automated? And how can we make it fair?
At SwampHacks, we hold two aspects of an application as the most important: experience and passion for tech. We still are not sure whether a more passionate group or a more experienced group creates a better hacker cohort.
The more passionate you are, the more likely you are to also be experienced. However, we like to have passion as an extra field because younger applicants can be very passionate but have limited experience in internships, research, projects, and so on. We judge an applicant's passion from their writing pieces in the application, such as essay responses.
Example:
Tell us about a project you have built and how you collaborated with others.
With these two quantifiers in mind, we asked organizers to rate each
applicant on a score from 1 to 5 for each aspect. These answers were saved
in our database and the application was marked as Reviewed.
Here I shall introduce the one, the only, BAT algorithm. It stands for Balanced Admissions Thresher. A thresher is a person or machine that separates grain from the plants.
The BAT algorithm is made of three primary parts: preprocessing, staging bucket isolation, and staged selection. I will go step by step. The implementation snippets and service code were written in Go.
Preprocessing applications includes assigning scores to applications, grouping teams, and doing other computations before moving on to the stage isolation step.
For each applicant i, we calculate a weighted score from the
scores given for passion and experience.
S_i = w_passion * s_passion + w_exp * s_exp + epsilonwhere:
w_passion + w_exp = 1.0s_passion and s_exp are each in [1, 5]epsilon = 0.1 prevents zero-valued scores, which is useful laterEach applicant is grouped into one of two buckets: individual applicants or teams. If an applicant was on a team, then we try our best to keep that team together, so teams get a dedicated selection bucket.
A team's weighted score is the average of all member scores. Teams are compared against other teams for admissions, and the selection details are specified below.
Per our head organizers, we have a few constraints and policies to consider. For example, we want a best effort for 70% of all acceptances to go to applicants from the University of Florida, with the remaining slots going to external schools. We also need to make a best effort for 65% of UF acceptances to come from what we consider early-career applicants, usually freshmen and sophomores.
When we structure these constraints, we form four dedicated selection pools that compete internally for spots:
This split keeps the selection process fair. Early-career applicants are, on average, less experienced than later-career applicants. If we compared everyone in one primary pool by weighted average, later-career applicants would tend to dominate. Isolating each group allows for fairer competition within each group.
To handle a lack of applicants in a specific pool, final slot allocations may roll over to a sister pool. For example, leftover allocations for UF Early may be given to UF Late, and vice versa. The same applies to the Other pools.
Finally, we begin the selection phase. To be completely fair, selection and bucket isolation are intertwined: bucket isolation does not happen until after the team phase of selection.
We use a modified version of Efraimidis-Spirakis random weighted sampling. This algorithm lets us randomly select applicants without replacement while still making weight significant in the outcome. That is useful because we want every applicant to have at least some chance of getting accepted, no matter how poor their application review was. It also reduces bias introduced by human reviewers.
k_i = v_i ^ (1 / w_i)where:
v_i is drawn uniformly from (0, 1) for applicant iw_i = S_i, the weighted score of applicant or team iS_i = w_passion * s_passion + w_exp * s_exp + 0.1k_i is the key used to rank applicants or teams for weighted sampling without replacement
Then we sort by key and select the top n applicants or teams
for that iteration.
Team selections happen first. We allocate a certain number of acceptance slots to team members. For example, if we allot 50 slots to teams and teams are capped at four members, this gives a minimum of 13 team slots:
50 // 4 = 12
50 % 4 = 2
=> 12 teams of 4 and 1 team of 2We apply Efraimidis-Spirakis random weighted sampling to each team's average score, then iterate through the selected teams and fill accepted slots until we run out of the team quota.
Afterwards, every non-selected team's members move into the individual applicant selection phase, giving them another chance to make it in based on individual merit and luck.
Finally, we loop twice over the buckets created above to cover rollover in
both directions between sister buckets. For each loop, we calculate each
individual's sort key using the random sampling algorithm, then select the
top n applicants, where n is the quota for that
bucket.
After all that mayhem, we were able to select 500 applicants for SwampHacks XI out of 1000+ applications with ease. We know this algorithm and process are not perfect yet, and there are many ways to expand it: adding an arbitrary number of scoring quantifiers, creating dynamic buckets from rulesets, and making the rollover logic more expressive. However, this current version lets us fairly accept applicants while giving everyone, no matter their score, a chance of getting in.
Thanks, alex out!