Abstract Background: Many junior doctors have poor stroke assessment skills. Although major efforts have gone in to changing the attitudes of clinicians to stroke through the development of guidelines and implementation strategies, the most important step may be to make sure that medical schools include appropriate teaching of this important topic in their curricula. The Rural Organization of Australian Stroke Teams Emergency Department (ROASTED) project sought to determine the effectiveness of a practical intervention to improve the assessment and education of stroke knowledge among our junior emergency department doctors. Methods: We used a prospective before and after study of two separate cohorts (intervention vs no intervention) to assess the stroke knowledge of our junior emergency department doctors and to test the effectiveness of an educational intervention. The project took place at five sites in rural Victoria in November 2006. Both cohorts undertook the same two validated quizzes 1 month apart. At the intervention sites two 1-h tutorials were conducted between the quizzes and participants were encouraged to use a web-based educational tool. Results: Pre-project stroke knowledge was shown to be poor at all of the participating sites. At the sites where no intervention took place no improvement in knowledge was shown (z = 0.83, P = 0.41, two-sided Mann–Whitney U-test). The median score for quiz 1 was 8.1 (41%, interquartile range (IQR) 6.5–9.4) and for quiz 2, 7.2 (36%, IQR 5.1–9.3). At the intervention sites, participants significantly improved their stroke knowledge between quiz 1 and quiz 2 (z= 4.75, P < 0.001). The median score for quiz 1 in this cohort was 8.3 (42%, IQR 6.5–10.0) and for quiz 2, 12.8 (64%, IQR 12.0–14.8). Conclusion: This project showed that junior doctors have an inadequate knowledge of stroke and that among our junior emergency department doctors there is a need for the ROASTED intervention and other innovative educational measures.