This study describes the context for the development of a tool to formatively assess information technology skills of students. The tool provides a reliable and valid assessment of word processing competency, utilizing automation to apply the test instrument via the Microsoft Office package. Tests can be designed directly by tutors, and delivered via a network. Evaluation of the tests suggests that immediate automated testing is preferred by students compared to a traditional written test. There is evidence that the tool improves the IT skills of its users, whereas a traditional written test has no such beneficial effect.