SciPath

Institutional memory

Common mistakes

Collected from mentors and judges after each season, organized by the stage where the mistake is still cheap to fix. Reading this needs no account, and none of it is specific to any one school.

55 entries 8 stages
01 · Forming the question 6
  • Picking a topic ("solar energy") instead of a question. A topic cannot be answered.
  • A question so broad that a funded lab would need five years for it.
  • A question already answered on the first page of Google Scholar.
  • Choosing based on the equipment available, then reverse-engineering a justification.
  • Confusing a demonstration ("build a solar oven") with an investigation ("does reflector geometry change peak temperature").
  • Beginning experimentation before the required forms are signed. This is unrecoverable and ends the entry.
02 · Prior art 6
  • Citing only sources that agree with you.
  • Citing news articles and blog summaries instead of the underlying paper.
  • Searching only the everyday term for the phenomenon and missing the field's actual vocabulary.
  • Reading abstracts only. The methods section is where the useful detail lives.
  • A literature review that is a stack of summaries with no argument about what is missing.
  • Not recording search terms and dates, so the search cannot be reproduced or updated.
03 · Design and methodology 9
  • Changing two things at once, so no result can be attributed.
  • No control, or a control that differs from the treatment in more than one way.
  • Pseudo-replication. Repeated measurements on one subject are not independent samples.
  • Deciding the outlier rule after seeing the data.
  • Choosing sample size by convenience and then not disclosing it.
  • Measuring what is easy instead of what the question requires.
  • No blinding when the measurement involves human judgment.
  • Engineering only, claiming improvement with no baseline measurement to improve on.
  • Engineering only, a single iteration. The rubric rewards iteration explicitly.
04 · Ethics, safety, and forms 6
  • Assuming a survey does not count as human participants research. It does.
  • Missing that work in a university or company lab triggers additional requirements.
  • Understating a mentor's contribution. Experienced judges detect this and it is an integrity issue, not a presentation one.
  • Overstating your own role in work that was substantially directed by someone else.
  • Forgetting the twelve-month research window.
  • Treating forms as paperwork to be done later. The dates are the point, not the signatures.
05 · Data collection 6
  • Recording only averages, making variance and error unrecoverable.
  • A single copy of the raw data.
  • Not logging anomalies at the time, then finding an unexplained pattern months later.
  • Letting the protocol drift over weeks without recording when it changed.
  • Running different conditions on different days without randomizing or blocking.
  • No record of instrument settings, so a measurement cannot be repeated.
06 · Analysis and interpretation 8
  • Claiming causation from correlation.
  • Running tests until one comes out significant.
  • Reporting a p-value with no effect size and no n.
  • Extrapolating beyond the range actually tested.
  • Treating a null result as a failed project and burying it.
  • Fitting a trendline to five points and reporting R-squared to four decimals.
  • A conclusion that restates the hypothesis instead of reporting what happened.
  • Excluding data points because they are inconvenient rather than by a pre-stated rule.
07 · Abstract, figures, and board 7
  • An abstract that says what you did but never says what you found.
  • Figures with unlabeled axes or missing units.
  • A board dense with text that nobody will stand and read.
  • Burying the result underneath the procedure.
  • Jargon used without definition.
  • Decorative chart styling, 3D pie charts, gradients behind data.
  • Font sizes chosen at a desk rather than tested from three feet away.
08 · Judging preparation 7
  • Memorizing a script and being unable to answer anything off it.
  • Claiming more certainty than the data supports.
  • Being defensive about limitations rather than owning them.
  • Being unable to explain your own analysis method in your own words.
  • Not knowing what your error bars represent.
  • Answering a question you were not asked because it is the one you rehearsed.
  • Talking for four minutes without pausing for the judge to redirect.

Where this comes from

Every entry was written down by someone who watched it happen. The list grows each March, right after judging, which is the only time anyone remembers this clearly.