By Dr. Bob Uttl (August 15, 2023)
How many licensed psychologists does it take to locate a correct test manual? At least six! The first three located the wrong one, the next two — the College of Alberta Psychologists’ (CAP) Deputy Registrar and the investigator contracted/hired by the CAP — did not try or did not locate any, and it is still unknown how many more it will take!
Dr. Bob Uttl
On September 15, 2010, Dr. Mary Westcott, Mandel and Associates Ltd., issued a report on her assessment of Ms. T. As part of the assessment, Dr. Westcott’s technician administered the General Aptitude Test Battery Canadian Edition (GATB CDN) (Nelson, 1986). Dr. Westcott did not cite nor mention in her report that Ms. T was administered the GATB CDN rather than the USES GATB (US DOL, 1970) developed decades earlier, in 1940s and 1950s, in the United States.. However, an examination of Dr. Westcott’s clinical files revealed Ms. T was administered the GATB CDN and her performance was scored using the GATB CDN General Working Population (GWP) norms.
In her September 15, 2010 report, Dr. Westcott described the GATB CDN administered to Ms. T. as follows:
General Aptitude Test Battery {GATB)
Dr. Westcott’s September 15, 2010 Report
The General Aptitude Test Battery (GATB) is the most widely-used aptitude test in North America. It consists of nine sub-tests that are grouped together into three major areas of functioning: cognitive abilities; perceptual abilities; and psychomotor abilities. Scores on the nine aptitude areas of the GATB are related to the requirements of specific jobs which are listed in the National Occupational Classifications (NOC), and as such it is possible to draw conclusions about the jobs a person might be capable of performing (from an aptitude point of view) by comparing GATB scores with aptitude requirements listed in the NOC. The GATB does what very few other tests do: it permits comparison of client characteristics with those of actual workers in specific occupations...
The words highlighted in blue are verbatim identical to the GATB description in Vespa v. Dynes, 2002 ABQB 25 (see para 137), the court’s decision published eight years earlier. The copied words apper in a section summarizing testimony of Dr. Michael Boissevain, Dr. Westcott’s colleague at Mandel and Associates Ltd.. Unfortunately, Dr. Westcott did not quote nor cite the source of the copied words in her report and presented them as her own.
Critically, Dr. Westcott’s largely plagiarized claim that “The GATB does what very few other tests do: it permits comparision of client characteristics with those of actual workers in specific occupations.” is patently false because (a) the GATB CDN has never been normed on any “actual workers in specific occupations,” and (b) the Career Handbook (Canada) and the National Occupational Classifications (NOC) do not provide any “requirements of specific jobs” and definitely not any “requirements of specific jobs” based on performance of any “actual workers in specific occupations.”
In 2021, Ms. T filed complaints against Dr. Mary Westcott, Dr. Allan Mandel, and Dr. J. Braxton Suffield with the College of Alberta Psychologists. One aspect of the complaints was that Dr. Westcott misused the Career Handbook Aptitude Levels, falsely claimed that they were “requirements” that Ms. T did not meet, and falsely claimed that they were based on “actual workers in specific occupations”; that Dr. Mandel reviewed Dr. Westcott’s report, claimed that it was “very well reasoned and based on very objective findings”, and failed to identify and correct Dr. Westcott’s misuses of the GATB, Career Handbook, and false statements; and that Dr. Suffield who was tasked to review conflicting reports by Dr. Westcott and Dr. K also failed to identifiy and correct Dr. Westcott’s misuses of the GATB, Career Handbook, and false statements.
In response to Ms. T’s complaints, the three psychologists — Drs. Westcott, Mandel, and Suffield — all wrote to the College of Alberta Psychologists and claimed that the GATB was in fact normed on “teachers”. To prove their claim, they referred to and copied page 170 of the GATB manual. However, they did not mention to the College of Alberta Psychologists that the page 170 they referred to was extracted from a wrong test manual, the test manual for a different test, the USES GATB (US DOL, 1970) rather than from the GATB CDN manual (Nelson, 1986) which does not even have page 170 and was published a decade and a half after the USES GATB manual.
On March 29, 2021, Dr. Westcott wrote to the CAP:
Upon review of the GATB manual, which I located in the in the Mandel and Associates test library and is the same manual used by Dr. Mandel in my training when I joined the firm, I can confirm that the GATB was normed on a sample of 234 elementary and secondary school teachers – the average G was 118 with a standard deviation of 13. A cross validation sample of 263 elementary and secondary school teachers indicated the average G to be 111 with a standard deviation of 13. Ms. T’s score on the G scale of the GATB was 83, over 2 standard deviations below the normative sample average. [REDACTED] Ms. T assert in [REDACTED] complaint that the GATB was not normed on actual workers. The GATB manual indicates otherwise and in fact was in part normed on a sample of teachers as reported on page 170. A copy of the relevant excerpt of the GATB Manual is appended as Appendix F.
Dr. Wescott’ March 29, 2021 letter to the CAP
On March 29, 201, Dr. Mandel wrote to the CAP:
The manual for the General Aptitude Test Battery (1979) provided a table (9.3) titled ”GATB Data on Aptitudes for Specific Occupations”. The table is 70 pages long and provides objective data on 446 occupations, including teacher (#404, page 170). Three sets of data for both elementary and high school teachers are provided: an initial validation sample (N=234); a cross validation sample (N=263), and a combined sample (N=497). Table 9.3 in the GATB manual shows that the mean scores on G (general learning ability – equivalent to FSIQ) were 118 (SD = 13) in the initial validation sample; 111 (SD = 13) in the cross validation sample, and 114 (SD = 13) for the combined sample…
Dr. Mandel’s March 29, 2021 letter to the CAP
…
The field testing with the GATB, included in Table 9.3 of the GATB manual, provided aptitude scores for teachers based upon a sample of 497 university seniors in Education programs. This found that the average score on G for the combined sample was 114 (SD=13). Ms. T’s score on this measure was 83, more than 2 SDs below the mean.
On May 24, 2021, Dr. Suffield wrote to the CAP:
She [Ms. T] further alleges that I then knew that no such comparisons were possible because, “no one actually administered GATB to any such actual workers in Ms. T’s occupation,” and that Nelson Canada had told me so.
This is incorrect, and disingenuous. As detailed in Sections III and IV of the Canadian edition of the GATB [emphasis added], many occupations – including elementary and secondary school teachers – were studied extensively when the GATB was developed. Table 9-3 (below) shows data from the 234 elementary and secondary school teachers who participated in an initial validation, and another 263 teachers who participated in a cross-validation study. Thus, GATB scores for teachers are based on a total sample of 497 teachers.
Dr. Suffield’s May 24, 2021 letter to the CAP
On September 27, 2021, Dr. Suffield wrote in his expert opinion letter filed in human rights proceedings:
8.2.5. Dr. Westcott, I, and others who conduct vocational and fitness-for-duty assessments use the GATB to determine aptitudes, and compare them to existing occupational data. The aptitude figures for elementary school teachers (NOC 4142) originated with 497 seniors in university education programs who participated in the initial and cross validation for this group. The data are in Table 9-3 of Section III (page 170) of the manual
Dr. Suffield’s September 27, 2021 Expert Report
for the General Aptitude Test Battery (GATB), and reproduced below.
8.2.6. Table 9-3 shows that the mean (“M”) scores on G were 118 (with a standard deviation (SD) of 13) in the initial validation sample, 111 (SD 13) in the cross-validation sample, and 114 (SD 13) for the combined sample of 497.
8.2.6.1. Ms. T’s score of 83 in Dr. Westcott’s 2010 assessment was more than 2 standard deviations below this average, at approximately the 0.9 percentile of this sample of nearly 500 prospective teachers.
Drs. Westcott, Mandel, and Suffield all located a wrong test manual. They located the USES GATB (DOL, 1970) manual reporting performance of some education university students in 1950s on a different test and at the time when only a small percentage of actual teachers had university degrees. Peformance of these students could never be considered “norms”, not in 1950s and not in 2010 or 2021.
Dr. Bob Acton from Falcongate Inc., the registered psychologist and investigator appointed by the College of Alberta Psychologist, also did not locate the correct test manual and stated that he and his colleagues lacked expertise in the area. Verbatim, on February 15, 2022, Dr. Acton wrote:
This investigation did not acquire any new information pertaining to Dr. Westcott’s use of the WAIS-R [sic], the GATB, or the MMPI-R [sic].
Dr. Bob Acton’s February 15, 2022 Investigation Report
These statistics and psychometric arguments along with the appropriateness of the test usage and interpretation is a highly complex and divisive issue that has been in both the academic literature and before the courts. An analysis of this complex issue is beyond expertise of this investigation.
On September 27, 2022, Dr. Troy Janzen, Complaints Director and Deputy Registrar, College of Alberta Psychologist, dismissed Ms. T’s complaints against Drs. Westcott, Mandel, and Suffield. With respect to the GATB, Dr. Janzen wrote:
My main reason to dismiss this allegation [Misuse and misinterpretation of the GATB and Career Handbook] is that Dr. Suffield was able, in my view, to provide an adequate response to the technical issues you raised. Dr. Suffield disputed your suggestion that he misused or misrepresented information either the GATB and Career Handbook. He was able to provide specific references from the test manual that supports his interpration and that of the other psychologist’s reported data. This included an excerpt fron the GATB Manual (p. 170) which clearly indicates that a sample of both elementary and secondary teachers were included and there was available data for comparision to your results….
Dr. Janzen’s September 27, 2022 Dismissal letter
Apparently, Dr. Janzen swalowed Drs. Westcott, Mandel, and Suffield’s false submissions to him hook, line and sinker. The “excerpt from the GATB Manual (p. 170)” provided to him by the three psychologists was the excerpt from the wrong test manual, the manual for a different test, the USES GATB (DOL, 1970) rather than the GATB CDN manual (Nelson, 1986). Dr. Janzen clearly did not locate the correct manual either. If he did, Dr. Janzen would know that the GATB CDN Manual does not have page 170. Similarly, if Dr. Janzen looked up the wrong test manual and examined its front page, Dr. Janzen would know it was the manual for a wrong test, published one a half decade before the GATB CDN was normed and manual published, reporting data on some non-representative sample of education students tested in 1950s, and provided no relevant comparison data whatsoever.
So, how many psychologists does it take to locate a correct manual? At least six. The first five — Dr. Westcott, Dr. Mandel, Dr. Suffield, Dr. Acton, and Dr. Janzen — either located a wrong one or did not locate any. Dr. Janzen’s dismissal of Ms. T’s complaints is under appeal before the College of Alberta Psychologists Complaints Review Committee (CRC). The CRC is composed of two licenced psychologists and two community members. A question arises: Will either psychologist on the CRC be able to locate a correct test manual and realize that it provides no data on any teachers?
Actually, the front pages of the USES GATB (DOL, 1970) and the GATB CDN (Nelson, 1986) Manuals were provided to the CRC as well as the link to the USES GATB (DOL, 1970) Manual. So the question is: Will the CRC members be able to recognize that Drs. Westcott, Mandel, and Suffield provided the College of Alberta Psychologists with the excerpt (p. 170) from the wrong test manual? Will they actually look where the “excerpt from the GATB Manual (p. 170)” came from?