Are psychologists able to correctly classify test scores? Can they create a reasonably accurate frequency table? Some can, some cannot, and some do not want to.

By Dr. Bob Uttl (October 5, 2023)

There are three kinds of lies: lies, damned lies, and statistics.

Attributed to Benjamin Disraeli by Mark Twain

In her response to Ms. T’s complaint against Dr. Mary Westcott to the College of Alberta Psychologists, on March 29, 2021, Dr. Westcott (from Mandel & Associates Ltd.) created a simple statistical summary of Ms. T’s scores — a frequency table titled “Summary of Ms. T’s scores obtained within my assessment”. The table gives the impression that all but three scores were “below 50th percentile” and that most scores were in “below the average range” including seven scores in the “impaired” range. Referring to the summary table she prepared, Dr. Westcott made the following misleading and false statements claims about Ms. T’s cognitive abilities and test performance:

My assessment indicated that the majority of her test scores fell below the 50^th percentile or within the impaired range (see table below).

Her only scores above the 50^th percentile included word reading and verbal delayed recall.

Dr. Westcott’s Resonse, dated March 29, 2021 (p. 12)

Dr. Mary Westcott’s summary table is copied below with marked-up text added in red. The red marked-up text next to various tests identifies numerous errors and falsehoods in Dr. Westcott’s table. Dr. Westcott’s table shows only three columns: one for “Scores below the average range”, one for “Scores in the average range but below 50th percentile”, and one for scores above the 50th percentile. Dr. Westcott did not explain what “Below the average range” means to her but, according to WAIS-IV/WMS-III/IV conventions, “below average” are scores corresponding to 70-79 on IQ/Index score scales (2 to 8th percentile or 2-8p) whereas scores below 70 IQ/Index scores (below 2nd percentile) are “exceptionally low” or “borderline” scores. Again, we have no idea what definition of “impaired” scores Dr. Westcott had in her mind when creating her table.

Dr. Westcott’s table yet again demonstrates Dr. Westcott’s astonishing inability or unwillingness to accurately present and classify Ms. T’s scores:

• Dr. Westcott falsely classified numerous “low average” scores as “below the average range” scores. Clearly, “low average” (“LA”) scores are still “average range” scores rather than “below average” range scores.
• Dr. Westcott falsely classified numerous “average” scores as “below the average range” scores.
• Dr. Westcott falsely classified even an “above average” score (GATB K) as an “below the average range” AND “impaired” score.
• Dr. Westcott omitted numerous “average” scores from her table including WMS – AD, WMS -VI, REY – Recognition, and numerous DKEFS scores.
• Dr. Westcott omitted numerous “Scores above the 50th percentile” including WMS – WMI, WRAT Reading, and nearly 20 KEFS scores.
• Dr. Westcott omitted all DKFES primary indexes (DKFES is said to measure executive functioning, according to Dr. Westcott). Notably, DKFES scores were mostly in the “high average”, “above average” and “exceptionally high” ranges and none were in “below average” or “exceptionally low” ranges.
• Whenever Dr. Westcott misclassified a score, Dr. Westcott mislabelled the score as being in a lower performance category than the score really was. Dr. Wescott misclassified 12 out of 18 scores shown in her “below the average range” category. In contrast, Dr. Westcott never misclassified a score into a higher performance category.
• Dr. Westcott included index scores that were lower but omitted index scores that were higher and/or highest. For example, for WMS-III, Dr. Westcott included AI (80/9p), IM (84/14p), ARD (75/5p), and GM (95/37p) indexes but omitted AD (94/34p), VI (112/79p), and WM (102/55p) indexes. Similarly, for WRAT, Dr. Westcott included SC (98/45p), Spell (97/42p), MC (Arithmetics) (90/25p), and RC (101/52p) indexes but omitted the highest index score — WR (105/63p). Similarly, for Rey Figure, Dr. Westcott included Copy (1p), IR (1p), and DR (1p) but omitted Recognition (46p)
• Dr. Westcott’s classification of GATB CDN scores is mostly false. For example, Dr. Westcott even classified GATB K as “impaired” when in fact it was “above average” (135g/96p) (Note that Dr. Westcott also failed to correctly score GATB CDN and contrary to her written statements she failed to add 1 SEM. See Errors, falsehoods, and plagiarism in scoring and interpreting psychological tests: Minimally competent conduct?)

If Dr. Westcott wanted to present Ms. T’s numerous scores classified into conventional categories, Dr. Westcott’s table would be very different and would, at one glance, demonstrate that Dr. Westcott presented a patently false picture of Ms. T’s performance to the College of Alberta Psychologists. The categories used in the table below are those used by WAIS-IV/WMS-IV/WMS-III, test manuals and are nearly identical to categories approved by the American Academy of Clinical Neuropsychology (AACN) (Guilmette et al., 2020). The two systems differ only in labels assigned to the extremely low or high scores: Whereas WAIS/WMS use “Borderline” label for scores below 70 and “Superior” for scores 130 or above, the AACN uses “Exceptionally Low” and “Exceptionally High”, respectively, for the same level of performance. The scores themselves were extracted from Dr. Westcott’s clinical file.

In the header of the table below, the first set of scores are index or IQ scores (M=100, SD=15), the second set of scores are percentiles identified by “p”, and the third set of scores are Wechsler scale scores with M = 10 and SD = 3 identified by “w”. Scores in bold are scores above 50th percentile (i.e., > 50p).

Exceptionally Low (Borderline) < 70 < 2p < 4w	Below Average 70-79 2-8p 4-5w	Low Average 80-89 9-24p 6-7w	Average 90-109 25-74p 8-11w	High Average 110-119 75-90p 12-13w	Above Average 120-129 91-97p 14-15w	Exceptionally High (Superior) ≥ 130 ≥ 98p ≥ 16w
GATB F RCFT Copy RCFT IR RCFT DR IVA VAQ	WMS-III ARD IVA FSAQ	WAIS-IV PRI WAIS-IV WMI WAIS-IV FSIQ WAIS-IV GAI WMS-III AI WMS-III IM GATB G GATB N GATB M IVA VRCQ IVA AAQ DKEFS TM VS DKEFS TM NLS DKEFS CWI I/S	WAIS-IV VCI WAIS-IV PSI WRAT-4 Spell WRAT-4 SC WRAT-4 MC WRAT-4 WR+ WRAT-4 RC+ WMS-III AD WMS-III VI WMS-III GM WMS-III WM+ WTAR SS+ RCFT Recog GATB V GATB S GATB P GATB Q IVA FSRCQ IVA ARCQ+ DKEFS VF CS:TCR DKEFS VF CS:TSA+ DKEFS CWI CN DKEFS CWI I DKEFS CWI CN+R+ DKEFS S SRDC DKEFS S CDS+ DKEFS WC TCC+ DKEFS T TAS+ DKEFS P TASFI	WMS-III VD DKEFS TM NS DKEFS TM LS DKEFS TM MS DKFS TM CN+LS DKEFS VF CF DKEFS DF EDO DKEFS DF S DKEFS CWI WR DKEFS S CCS DKEFS S FSDC	GATB K DKEFS DF FD DKEFS DF TC DKEFS DF CFED DKEFS DF CF+ED	DKEFS P TASMC DKEFS VF LF
Count
5	2	14	29	11	5	2

Examination of the actual scores in Dr. Westcott’s clinical file reveals that Dr. Westcott’s presentation of Ms. T’s cores and Dr. Westcott’s claims about Ms. T’s performance were stunningly false:

• Dr. Westcott’s table summarising Ms. T’s score misclassified Ms. T’s scores as lower than they were and omitted numerous high scores (see above).
• Dr. Westcott’s claim that “the majority of her test scores fell below the 50^th percentile or within the impaired range” is technically correct but stunningly misleading. If Dr. Westcott tested 100 examinees of average abilities, for about 50 of them — “the majority” of the scores — would fall under the 50th percentile, by definition.
• Dr. Westcott’s claim that “Her [Ms. T’s] only scores above the 50^th percentile included word reading and verbal delayed recall” is again stunningly false. It deserves at least 25 Pinocchios, at least one for each score above the 50th percentile (over 25 scores in total) that Dr. Westcott did not disclose in her table. For comparison, according to scientists, Pinocchio’s neck would have broken after only 13 lies.

Dr. Westcott’s presentation of Ms. T’s scores cannot be explained by random incompetence, negligence, or even recklessness. Random incompetence, negligence, or even recklessness would result in Dr. Westcott misclassifying scores sometimes into lower and sometimes into higher performance categories, and omitting some low scores and omitting some high scores. Dr. Westcott’s misclassification of a large number of scores into lower performance categories, omission of a large number of high scores, and stunningly false claims that there were only TWO scores above 50th percentile (even though even Dr. Westcott’s table showed three such scores and even though there was nearly 30 of such scores) is extremely unlikely. The probability that Dr. Westcott misclassified 12 scores into lower performance categories and omitted over 20 “low average”, “average”, “high average”, “above average” and “exceptionally high” due to mere incompetence, negligence, or recklessness is less than 1 in 1,000,000. Thus, on the balance of probabilities, Dr. Westcott maliciously misclassified Ms. T’s scores as lower than they were, omitted a large number of Ms. T’s high scores, and lied to the College of Alberta Psychologists about Ms. T’s scores in order to paint Ms. T as cognitively impaired. In fact, it is nearly certain given less than 1 in 1,000,000 chance that this level of misclassifications, omissions, etc. would occur by random chance, through incompetence only.

Dr. Troy Janzen, Deputy Registrar and Complaints Director, College of Alberta Psychologists, swallowed Dr. Westcott’s stunningly false presentation of Ms. T’s scores hook, line, and sinker. Referring to the 18 scores classified by Dr. Westcott as in “below the average range” in her Table 1, Dr. Janzen wrote:

Dr. Westcott also objected to the repeated characterisation of your overall cognitive results as being “average”. She notes that the results from her own assessment [while Ms. T was physically ill, vomited, etc.] showed a full scale IQ score in the low average range… She notes that her conclusion about cognitive decline … was drawn from a low average performance on a wide range of measures. Dr. Westcott noted that you scored in this same low average range on 18 different measures/subtests that were given to you at the time…

Dr. Janzen, Dismissal of Ms.T’s Complaints, dated September 27, 2022 (p. 13)

A few paragraphs later, Dr. Janzen claimed that he “personally reviewed all the testing completed by Dr. Westcott’s assistants”. Verbatim, Dr. Janzen wrote:

I personally reviewed all the testing completed by Dr. Westcott’s assistants to ensure that all raw scores were correctly summed, that the psychometricians followed reported basal and ceiling rules as required in standardized administration, that scores and responses were faithfully recorded in the protocols, and that scores were correctly reported. I can attest that scoring was all done accurately with no errors in administration that I could detect…

Dr. Janzen, Dismissal of Ms. T’s Complaints, dated September 27, 2022 (p.14)

In the next paragraph, Dr. Janzen continued:

… My fulsome review of the totality of Dr. Westcott’s assessment work supports the conclusion that the answer to all these questions is affirmative. That is, she used appropriate methods including a wide range of reliable and valid assessment tools. Her interpretations were within an acceptable range based on the totality of the data. Her administration and scoring of all tests appeared to fall within acceptable ranges of standardization and were accurate. She obtained multiple sources of data in coming to her conclusion. Thus, the evidence does not support your allegations that Dr. Westcott demonstrated incompetence.

Dr. Janzen, Dismissal of Ms. T’s Complaints, dated September 27, 2022

Dr. Janzen did at minimum a very sloppy job when he “personally reviewed all the testing” as he did not notice that Dr. Westcott’s description, scoring, and interpretation of Ms. T’s GATB CDN was full of errors, falsehoods, and plagiarism (see Errors, falsehoods, and plagiarism in scoring and interpreting psychological tests: Minimally competent conduct?). Dr. Janzen also did not notice that Dr. Westcott was using decades-obsolete GATB CDN “norms” from 1985 (see Using outdated, obsolete, and irrelevant test data to make disparaging statements about a client’s IQ, intelligence, and cognitive abilities: Minimally competent practice, unprofessional conduct, and/or malpractice). Similarly, Dr. Janzen’s “fulsome review of the totality of Dr. Westcott’s assessment” was not likely very “fulsome” as Dr. Janzen missed numerous errors, falsehoods, and plagiarism in Dr. Westcott’s work.

Nevertheless, to the extent to which Dr. Janzen actually “personally reviewed all the testing” and did “fulsome review of the totality of Dr. Westcott’s assessment”, Dr. Janzen knew that numerous scores were missing from Dr. Westcott’s table summarising Ms. T’s scores. For example, Dr. Janzen knew that all of the DKEFS tests scores were omitted in Dr. Westcott’s summary, that WMS AD, WMS VI, and WMS WMI were omitted, etc.. Moreover, if Dr. Janzen actually checked the veracity of what Dr. Westcott submitted to him, Dr. Janzen would have to conclude (assuming that he is at least minimally competent in classifying test scores into categories himself) that Dr. Westcott’s submissions to the College of Alberta Psychologists were stunningly false.

Why did Dr. Janzen swallow Dr. Westcott’s stunningly false submissions hook, line, and sinker? Did he not “personally reviewed all the testing”? Did he not do the “fulsome review of the totality of Dr. Westcott’s assessment”? Did he decide to close his eyes to Dr. Wescott’s stunningly false submissions to him? Did he not have sufficient memory capacity to remember that there were many more scores to be summarised in Dr. Westcott’s table? Did Dr. Janzen lack essential competence to realize that Dr. Westcott misclassified and omitted numerous scores? Was Dr. Janzen focused on protecting Dr. Westcott rather than the public as required by Alberta’s Health Profession Act? Did Dr. Janzen intend to sweep Ms. T’s complaints under the proverbial rug?