Overview
Award deliberation is the final step in the judging process. During this step, judges work with the judge advisor and each other to select candidates for each award and develop a plan for gathering any follow-up information needed to make final decisions.
Award deliberations involve comparing teams with one another. The integrity of the judging process depends on judges being able to speak candidly during this process. The information discussed during deliberations is particularly sensitive, so all deliberation notes and conversations must remain confidential during and after the event.
The applicable rubrics are tools to assist with deliberations. A team’s score—whether for a specific line item on a rubric or the overall score—is one data point that judges and the judge advisor can consider as part of the decision-making process. Scores are not a substitute for qualitative judgment. Judges should also avoid over-relying on drone performance when selecting award winners or finalists. Some awards directly reference teams’ match performance and rankings, while others may consider drone performance indirectly or use it as a point of comparison. Because circumstances vary from event to event, qualitative judgment and consensus are the best tools for determining how performance metrics should factor into award decisions.
Step 1 – Award Nominations from Each Judge Group
After each group of judges has completed their assigned interviews, they should identify one or two teams from their group as strong candidates for each applicable award. It is not necessary to nominate a team for every award. Once selections are made, judges should return to the judges’ room to share their nominations with the full judging team and the judge advisor. A common method is to write team numbers on sticky notes and place them under printed award titles on a wall or board, allowing all judge groups to view and discuss the nominations collaboratively.
Award Descriptions and Criteria can be printed and used to help visually organize judge input and candidate teams during deliberations. Color coding using sticky notes can help keep the nominations from each judge group organized.
The end result will be a shortlist of nominations for each award from all judge groups. When there are many award nominations for an award, the judge advisor may ask judge groups to withdraw weaker candidates from consideration based on brief arguments for and against each nomination. Alternatively, a judge group, upon considering the merits of other candidates, might withdraw their nomination for their initial candidate.
Step 2 – Follow-Up Interviews for Award Nominees
This step should be completed before the end of alliance/teamwork matches. The judge advisor will organize judge groups to gather further information to validate the award nominees. This may take the form of observing solo matches, alliance/teamwork matches, and behavior in the pits, as well as conducting follow-up interviews with award nominees. The goal is to determine a final ranking of nominees for each award being presented.
For follow-up interviews, it is recommended that the nominees are interviewed by judges who have not interviewed them previously. If possible, group judges who share an area of expertise to evaluate particular awards. For example, judges who have a background in programming or computer science would likely be best qualified to evaluate the finalist nominees for the Coding Award in ADC and the Coding Systems Award in ADC Pro. This guidance specifically differs from initial interviews, in which judges with similar expertise should be assigned to different judging groups with the intent of giving all teams a more well-rounded initial assessment.
Teams should not be told which awards they are being considered for, as this violates the confidentiality principle outlined in the Drones Guide to Judging.
Step 3 – Final Ranking and Nominations
The next step is the final deliberation for each award being given at the event. This should be completed shortly after the start of the Elimination Matches. Quantitative data (Alliance/Teamwork Rankings and Solo Missions Rankings) are needed for deliberations for certain awards and can be obtained from the event management software, Competition Control/C².
After the follow-up interviews are conducted, the judges who participated should be allowed to deliberate and rank the teams interviewed. As a best practice, judges should identify a first-choice award nominee, along with three or more additional alternate candidates. The Final Award Award Nomination Ranking Sheet provides a way to list the ranked awards and candidates for consideration. The ranked list should go deep enough so that if a top candidate were to be chosen for a higher award or removed from eligibility, there would still be enough alternative candidates to choose from.
In the case of the All-Around Champion Award, the winner should come from the list of Communications Mission Award nominees/finalists for ADC and from the Engineering Design Award nominees/finalists for ADC Pro that meet the criteria for Performance Awards and other Judged Awards. Moving a team from being a Communications or Engineering Design Award finalist to All-Around winner may result in a reshuffling of winners for other awards to ensure that no team earns more than a single judged award at the event. The judge advisor should reconcile award winners to ensure that each award winner earns the highest award for which they are eligible at the event. Having three or more ranked candidates for each award is very helpful in this situation and eliminates the need for additional deliberations.
- If information arises indicating a possible violation of the Code of Conduct or Student-Centered Policy, the judge advisor must review the team’s eligibility for judged awards. If the concern is substantiated, the award should be given to the next alternate in the ranking. For egregious violations, the judge advisor should consult the event partner and RECF Regional Support Manager.
Step 4 - Entering of Award Winners into Competition Control/C².
After award winners have been determined, the judge advisor should inform the event partner that the process is complete. The winning team numbers are entered in the Award Winner Selection section of Competition Control/C². It is recommended to print the Award Summary Sheet or Award Script Reports, so the judge advisor can double-check that all award winners have been entered correctly.
Step 5 – Collection and Treatment of Judging Materials
Prior to the award ceremony, the judge advisor should secure the judges’ room, including collecting all notes, rubrics, and ranking sheets, and erasing any whiteboard notes. Judges should not retain copies of any notes that reference individual teams, including rubrics or award ranking sheets. If pictures of teams or machines were taken, judges should delete them.
After the event is over, the judge advisor should destroy all judging materials off-site. These items are not to be given to the event partner for disposal.
Continue to Next Section