2a:null 2b:["$","title",null,{"children":"The Reliability of Grading Writing using the Teacher Assessment Framework | No More Marking | Observable"}] 30:T796,md` In England, the writing of pupils at the end of their Primary phase of education is graded by teachers using the Teacher Assessment Framework [TAF](https://www.gov.uk/government/publications/teacher-assessment-frameworks-at-the-end-of-key-stage-2). Using the TAF rubric, the teachers grade students with three grades: Greater Depth Standard (GDS), Expected Standard (EXS), Working Towards Standard (WTS). The teachers consider a range of writing produced by their students and decide on a grade. To study the reliability of the TAF grading process [No More Marking](https://www.nomoremarking.com/?utm_source=blog&utm_medium=referral&utm_campaign=taf/) recruited 30 teachers experienced in using the TAF to grade a sample of 349 scripts. Despite being experienced, all teachers were required to undergo further training from a moderator with experience as a Local Authority Lead Moderator and as an Operational External Moderator and Deputy Standardisation Team Leader for the national Standards and Testing Agency. The following interactive notebook shows the scripts that were graded along with the range of grades given by the teachers. You can see details of the task and the conditions [here](https://blog.nomoremarking.com/assessing-primary-writing-2019-20-year-6-results-f1628004e7d3). In addition to the grading by the TAF, all scripts were judged using Comparative Judgement (CJ) as part of the national [Assessing Primary Writing](https://www.nomoremarking.com/?utm_source=blog&utm_medium=referral&utm_campaign=taf) project. The grade assigned by the CJ process is provided in the table. The research concludes that the CJ approach achieves twice the classification accuracy of the TAF approach, in the same amount of time. You can read the report summary [here](https://www.nomoremarking.com/?utm_source=observable&utm_medium=referral&utm_campaign=taf) and the full report [here](https://osf.io/preprints/socarxiv/vzus4). `31:[{"id":"27a53021acb0fcf791267a699a0e2d5c86d6a1da503e293e1dd172010052b1c4e12c4afae29feef77c9547b8cefc7947a861a42735041c1de1c00c5a12456115","href":"https://static.observableusercontent.com/files/27a53021acb0fcf791267a699a0e2d5c86d6a1da503e293e1dd172010052b1c4e12c4afae29feef77c9547b8cefc7947a861a42735041c1de1c00c5a12456115","name":"grade-ranges-3.csv","type":"text/csv","lastModified":1720454084041.627,"size":102144},{"id":"33f20ce680ffeb7df8ab9594097e59397830ad9bb1dd72dd0fc380f2345f380bbef9ef286f50f6e132a74311c656b287b693239530b737358e135edde46da4b8","href":"https://static.observableusercontent.com/files/33f20ce680ffeb7df8ab9594097e59397830ad9bb1dd72dd0fc380f2345f380bbef9ef286f50f6e132a74311c656b287b693239530b737358e135edde46da4b8","name":"grade-ranges@2.csv","type":"text/csv","lastModified":1599145811129.464,"size":139480}] 2e:{"id":"92e67cba76805450","slug":"the-reliability-of-grading-writing-using-the-teacher-asses","title":"The Reliability of Grading Writing using the Teacher Assessment Framework","thumbnail":"ef6e1d2afdad7a58fb6a70a7f128aadf9e6bd435ef493d533e666277a2359cb3","access_level":"public","create_time":"$D2020-09-03T13:12:08.077Z","update_time":"$D2024-07-08T15:57:50.968Z","view_time":null,"first_public_time":"$D2020-09-14T08:08:31.045Z","has_unpublished_edits":null,"fork":false,"stars":1,"starred":false,"owner":{"type":"team","id":"b86f12f48a31a9a8","login":"nomoremarking","name":"No More Marking","avatar_url":"https://avatars-next.observableusercontent.com/a0a44b016e20e321c076294ee6a9a440b366260678320f2c21f83e5d58803730","tier":"starter_2024"},"version":170,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"7b72bccfd1e1b45b","login":"nmm","name":"Dr Chris Wheadon","avatar_url":"https://avatars-next.observableusercontent.com/2eeb4f8083f36b6f35ccf1379d8decd22194ecdc0e3840927d71a41f0bd1762b","tier":null}],"members":[{"type":"individual","id":"7b72bccfd1e1b45b","login":"nmm","name":"Dr Chris Wheadon","avatar_url":"https://avatars-next.observableusercontent.com/2eeb4f8083f36b6f35ccf1379d8decd22194ecdc0e3840927d71a41f0bd1762b","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":{"id":"143db16b69cf6fbe","slug":"an-introduction-to-comparative-judgement","title":"An Introduction To Comparative Judgement","owner":{"id":"b86f12f48a31a9a8","login":"nomoremarking","name":"No More Marking"}},"type":"notebook","body":{"title":"The Reliability of Grading Writing using the Teacher Assessment Framework","cells":[{"id":0,"value":"md`# The Reliability of Grading Writing using the Teacher Assessment Framework`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":82,"value":"$30","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":109,"value":"md`Use the button below to view the scripts and the range of grades given by the teachers.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":14,"value":"viewof i = Scrubber(numbers, { delay: 10000 }, { autoplay: false })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":45,"value":"md`### Grades`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":119,"value":"md`The table shows the number of teachers giving the script each of the grades, along with the median grade and grade range according to the TAF. To provide more nuance to the grading the teachers were asked to give a fine grade, breaking each wide grade into A, B and C. The fine grades are not part of the national writing moderation system.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":41,"value":"html`\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

WTSC	WTSB	WTSA	EXSC	EXSB	EXSA	GDSC	GDSB	GDSA	TAF Grade Range	Median TAF Grade	CJ Grade
${script.WTSC}	${script.WTSB}	${script.WTSA}	${script.EXSC}	${script.EXSB}	${script.EXSA}	${script.GDSC}	${script.GDSB}	${script.GDSA}	${script.range}	${script.medianLabel}	${script.cjGrade}

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":129,"value":"md`\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":99,"value":"md`### Script:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":27,"value":"html`

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":133,"value":"md`\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":4,"value":"md`## Appendix`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":62,"value":"html`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":17,"value":"numbers = Array.from({ length: data.length }, (_, i) => i)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":6,"value":"data = d3.csvParse(await FileAttachment(\"grade-ranges-3.csv\").text())","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":25,"value":"script = data[i]","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":28,"value":"page1 = script.page1","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":31,"value":"page2 = script.page2","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":7,"value":"d3 = require(\"d3@6\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":12,"value":"import { Scrubber } from \"@mbostock/scrubber\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W31","stdlib":"1","titling":"auto"}} 2f:"$2e:body"