Judging

How We Judge

ASERS judging was shaped by our own experience competing in major science fairs and research programs, and by research on how judges evaluate complex work.

Research on expert grading has found that judges can form an overall impression of a project and then give similar scores across several supposedly separate rubric categories. Research on review panels has also shown that discussion can help judges share expertise, while sometimes making individual opinions less independent.

Three-part evaluation

Oral and poster judges submit scores independently before seeing the other group's evaluations.

What judges look for

• Scientific quality: Are the methods, controls, analysis, and study design sound?
• Evidence: Do the results support the conclusions?
• Original contribution: What did the project add?
• Understanding and ownership: Can the student explain the choices, results, limitations, and their own contribution?

Project size, expensive equipment, large datasets, complicated models, polished graphics, and mentor prestige do not carry weight on their own.

Judges also answer two explicit questions: What intellectual contribution did the student make? Does the conclusion follow from the evidence actually generated?

Scoring and deliberation

Judges use a short rubric based on the four areas above. We keep the rubric short because adding more scoring categories does not necessarily make judgments more independent.

Independent scoring → score submission → judge discussion → optional private revision

Judges discuss projects only after their first scores are submitted. Discussion gives judges a chance to raise scientific concerns, explain field-specific details, and point out things another judge may have missed. Judges do not have to reach consensus.

Judge assignment and conflicts

Projects are assigned centrally based on relevant expertise and conflicts of interest. Judges may not evaluate students they have mentored, supervised, collaborated with, or have a close personal or professional connection to. Conflicts can also include a connection to a direct competitor.

Parents or guardians of competitors may not take part in judging decisions that could affect their child's awards.

Reducing bias

Our goal is to reduce how much any one person's impression can affect the final result. ASERS uses:

• multiple judges for each project
• separate oral and poster panels
• independent scoring before discussion
• central judge assignment
• conflict-of-interest screening
• review of major scoring disagreements
• paper review focused on scientific consistency rather than writing polish
• private final rankings after discussion

Feedback that helps researchers improve

A judging interaction should remain useful after the rankings are published. ASERS judges are asked to identify the strongest part of the work, an important limitation, an alternative explanation, one way to strengthen the conclusion, and a question worth pursuing next.

Judge training

Before the event, judges receive short guidance on following the evidence closely, asking follow-up questions based on the student's answers, checking whether claims match the data, separating future potential from demonstrated results, evaluating student understanding and contribution, recognizing limitations as part of good scientific reasoning, and using discussion to share expertise without pressuring other judges to agree.

Final awards

For top awards, judges first rank finalists independently. They then discuss the projects, raise scientific concerns, explain field-specific details, and resolve factual questions. After discussion, each judge submits a final private ranking.

The final decision uses those rankings along with the oral and poster results and any important concerns raised during paper review.