[EDIT: The article is now available online, open access(!) via Project MUSE: https://muse.jhu.edu/article/725984/pdf]
Following four years of work [EDIT: I realized it’s actually been 6 years!], a project I did with several colleagues has just been accepted for publication in the journal Language. Together with Taylor Jones (U Penn Linguistics, www.languagejones.com), Ryan Hancock (Philadelphia Lawyers for Social Equity, WWD), and Robin Clark (U Penn Linguistics), I conducted a study to test Philadelphia court reporter’s transcription accuracy and comprehension of African American English (see Taylor Jones’ explainer on this language variety).
In order to work as an official court reporter in Philadelphia, court reporters have to be certified by the court system at 95% accuracy. In other words, they have to be able to correctly transcribe 95% of all words they hear in a test at specific speeds (words per minute) based on the type of speech — there are different required speeds for testimony versus question and answer, etc. However, based on informal interviews with court reporters, we determined that their certification tests and training are based on Mainstream American English spoken by lawyers, judges, and broadcasters. We know from previous research that there are issues of cross dialect comprehension between speakers of AAE and MAE (some examples outlined here) so we decided to test court reporters’ ability to accurately transcribe AAE as spoken by native speakers from the Philadelphia area.
We recruited 9 native speakers of AAE (5 men and 4 women of varying ages) from West Philadelphia, North Philadelphia, Harlem, and Jersey City and recorded them reading 83 sentences in AAE that were chosen from actual heard speech (we did not create the sentences from imagination). These sentences included syntactic features of AAE both alone and in combination. We then randomized the voices and sentences. We played the audio for 27 court reporters — one third of the official court reporting pool in Philadelphia. Reporters were given a 220Hz warning tone followed by a sentence repeated twice and then ten seconds of silence. The sentences were played at 70-80 decibels at 10 feet — more than loud enough for the court reporters to hear — and at speeds much slower than their certification tests. We asked the court reporters to transcribe the sentence and then paraphrase the sentence in “classroom” English. While we were aware that paraphrasing is not part of their normal job, we were curious if miscomprehension contributed to mistranscription.
The results showed that the court reporters in our sample could not transcribe spoken AAE at their required level of accuracy. When measured at the sentence level - was the sentence right or wrong - our sample transcribed 59.5% of sentences accurately on average. When measured at the word level, on average our court reporters transcribed 82.9% of the words accurately. In 31% of transcriptions, errors change who, what, when, or where. Accuracy was not related to race, age, where they got their training, or the number of years on the job. Additionally, court reporters paraphrased the sentences correctly only 33% of the time. Surprisingly, reporters’ individual paraphrase and transcription accuracy were not systematically related.
From post-test conversations with the study participants, it was clear that these court reporters’ wanted the tools to perform better and did not hold explicit malice towards the speakers or the individuals they transcribe in court. They did, however, express the opinion that speakers like the ones we played for them were speaking incorrectly and that the difficulties they had with transcription were the fault of the speaker. Rather, in our paper, we contend that court reporters are not given appropriate training related to other varieties of English they are likely to encounter in their day-to-day work. Given that the court reporter is responsible for the official court record and that the official record has consequences in terms of cross-examination, appeals, etc, it would behoove the court system to ensure that certification standards are related to the task at hand.
You can read more here on the study, its implications, and what we think comes next.