This article reports on the project to design and implement a web-based mechanism for formative self-assessment, to be used as part of Mick Short's online stylistics course, Language and Style (see the preface to this issue for details). Using the self-assessment mechanism, students can compare their own performance on extended academic tasks (the stylistic analysis of three literary texts – a poem, a piece of prose fiction and a drama extract) with benchmarked sample answers. In this article we describe the process of developing benchmarks for the self-assessment mechanism and explain how this mechanism works in practice. Finally, we discuss students’ reactions to the self-assessment instruments, as part of our own assessment of their validity and pedagogical value.