Using outdated, obsolete, and irrelevant test data to make disparaging statements about a client's IQ, intelligence, and cognitive abilities: Minimally competent practice, unprofessional conduct, and/or malpractice? - School District No 5 Southeast Kootenay Information

By Dr. Bob Uttl (August 15, 2023)

Recently, the three registrants of the College of Alberta Psychologists — Drs. Mary Westcott, Allan Mandel, and J. Braxton Suffield — relied on several sets of decades outdated, obsolete, and irrelevant test data to make disparaging statements about Ms. T’s IQ, intelligence, and cognitive abilities. They claimed that Ms. T’s twice-assessed average intelligence prevented her from performing her teaching duties. Drs. Westcott, Mandel, and Suffield directly compared:

(1) Ms. T’s Wechsler Adult Intelligence Scale IV Canadian (WAIS-IV CDN) (Wechsler, 2008) FSIQ scores to the mean WAIS (Wechsler, 1955) IQ scores of some teachers somewhere as if the scores on the two Wechsler tests published 53 years apart were equivalent and sample of teachers tested decades ago representative of teachers in the School District No. 5 Southeast Kootaney today or in 2010.
(2) Ms. T’s WAIS-IV CDN (Wechsler, 2008) FSIQ scores to the mean General Classification Test (GCT) (US Adjutant General, 1941) standard scores of some teachers somewhere, White enlisted men in US Army in 1940s at the time of the World War II, as if the scores on the two tests published nearly 70 years apart were equivalent and White enlisted men in World War II representative of teachers the School District No. 5 Southeast Kootaney today or in 2010. The GCT data on White enlisted men were originally published by Harrell and Harrell (1945) and republished by Schmidt and Hunter (2004).
(3) Ms. T’s GATB CDN (Nelson, 1986) Standard Scores to the mean USES GATB (1970) Standard Scores of some university students in the final year of their university education degree programs in 1950s as if the scores on the two tests were equivalent and the students representative of teachers. These students were of course not representative of teachers because only a small percentage of teachers in 1950s had university degrees.
(4) Ms. T’s WAIS-IV CDN (Wechsler, 2008) FSIQ scores to Gottfredson’s Occupation Table based on Wonderlic Personnel Test (Wonderlic, 1992)/ WAIS (Wechsler, 1955) IQ scores as if the two Wechsler tests published 53 years apart were equivalent and sample of unknown teachers representative of teachers in the School District No. 5 Southeast Kootaaney today or in 2010, and
(5) Ms. T’s WAIS-IV CDN (Wechsler, 2008) FSIQ scores to Gottfredson’s Wonderlic Personnel Test (Wonderlic, 1992)/WAIS (Wechsler, 1955) means scores of “college format”. According to Gottfredson, the “college format” had WAIS (Wechsler, 1955) FSIQs ranging from 113 to 120.

These comparison are unwarranted, unscientific, and plainly wrong for numerous reasons, including but not limited to the following:

The tests are not equivalent and, therefore, their scores cannot be directly compared. If these psychologists wanted to speculate as to what Ms. T’s scores on WAIS (Wechsler, 1955), GCT (US Army Adjutant, 1941), USES GATB (US DOL, 1970) were given her scores on WAIS-IV CDN (Wechsler, 2008) FSIQ and GATB CDN (Nelson, 1986), the three psychologists would have to have some regression equations allowing them to calculate Ms. T’s estimated scores on these tests she was never administered. The three psychologists did not seem to have and did not use any such regression equations. They compared the scores directly as if they were equivalent.
It is well established that the intelligence of populations have been rising by about 0.3 IQ points per year. This generational increase in IQ is called the Flynn Effect. Thus, the three psychologists would need to at minimum adjust Ms. T’s WAIS-IV CDN (2008) FSIQ for Flynn Effect before they could speculatively compare Ms. T’s WAIS-IV CDN FSIQ to the mean WAIS (Wechsler, 1955) scores of some samples of some teachers somewhere. Accordingly, if they wanted to compare Ms. T’s WAIS-IV CND FSIQ of 91 (Dr. K’s assessment), they would need to add approximately 0.3 x (2008 – 1955) = 15.9 IQ points to Ms. T’s WAIS-IV CDN score before they could speculatively compare it to some groups of some teachers tested with WAIS (1955). It is a well-known scientific fact that as a result of the Flynn Effect, more recent WAIS tests are more difficult than earlier WAIS tests.
It is also well established that to make claims and pronounce opinions about how Ms. T’s intelligence compares to her School District No. 5 Southern Kootaney peers, or for that matter to elementary school teachers in BC in 2010, the three psychologists would need to have some normative data on how the teachers in School District No. 5 Southern Kootaney or BC perform on the same test. The theee psychologists do not seem to understand this basic requirement and instead used non-representative samples of some teachers tested in US decades (up to 80 years) ago. To spell it out: (1) White enlisted men tested in 1940s with GCT were not representative of teachers in 1940s nor today; they were not representative of teachers in the School District No. 5 Souther Kootaney nor of teachers in BC in 2010, (2) Students in their final year of education university degree in 1950s tested with the USES GATB (US DOL, 1970) were not representative of teachers in 1950s, nor of teachers today. In 1950s, these students were elite, privileged few who actually were about to attain their undergraduate degrees in a historical era where a small portion of teachers attained university degrees. (3) Nothing is known about five hundred teachers tested by ten corporations with Wonderlic Personnel Test (Wonderlic, 1992). We do not even know whether they were elementary school teachers, high school teachers, or college teachers. Clearly, there is no evidence that these Wonderlic (1992) teachers were representative of anyone.
It is also self-evident that psychologists who wish to claim that someone does not meet the minimum intelligence requirement of her job and that Ms. T’s twice assessed average intelligence and cognitive abilities “prevent” Ms. T from performing her teaching duties (at elementary not rocket-science university level) ought to know what those minimum requirements, for example, minimum WAIS-IV CDN FSIQ, are. The three psychologists never obtained any such minimum IQ and other cognitive ability requirements from the School District District No. 5. In fact, Mr. Frank Lento, the Chair of the Board of the School District No. 5 already testified that the School District No. 5 had no minimum IQ, intelligence, cognitive ability, or even personality requirements.
With respect to Gottfredson’s “college format” having WAIS (Wechsler, 1955) IQs between 113 to 120, this used to be the case back in 1950s but is false today. First, Longman et al. (2007) re-analyzed WAIS III US (1997) and WAIS III CDN (1997) normative data and reported the following: (a) US college students (i.e., those with 13 to 15 years of education) had mean FSIQ of only 103 and the FSIQ of the middle 96% ranged from 77 to 128.; (b) US university graduates (i.e., those with 16 or more years of education) had mean FSIQ of only 112 and the FSIQ of the middle 96% ranged from 86 to 140; (c) Canadian college students had mean FSIQ of only 104 and the FSIQ of the middle 96% ranged from 74 to 130; and (d) Canadian university graduates (including those with PdDs) had mean FSIQ of only 109 and the FSIQ of the middle 96% ranged from 78 to 142. Second, Holdnack and Weiss (2013) reanalyzed WAIS-IV US (2008) normative data and found the following: (a) US college students FSIQ declined still further, down to FSIQ of 101.4 (SD = 13.1) for those with 13 to 15 years of education; and (b) US university graduates’ (those with 16 years of education) FSIQ declined to 107.1 (SD = 14). To my knowledge, no analyses of WAIS IV CDN (2008) normative data were published to date. However, based on the WAIS-III (1997) US vs. CDN comparison, it is likely that Canadian college students’ mean WAIS-IV CDN dropped down to 100 and that Canadian university graduates mean WAIS-IV CDN FSIQ dropped down to 104 IQ points or even lower. Gottfredson’s (1997, 1998, 2003) claims about IQ of “College Format” were outdated, obsolete, and false by the time she published them in 1997, 1998, and 2003, not surprisingly, since Gottfredson relied on data published by Wonderlic (1992).

Critically, numerous ethics codes are clear that psychologists shall not base their opinions and recommendations on data or test results that are outdated, obsolete, and not useful (e.g., APA Ethical Principles of Psychologists and Code of Conduct, Section 9.08; College of Psychologists British Columbia Code of Conduct, Section 11.21). The College of Alberta Psychologists does not mention obsolete tests in the Standards of Practice but in the Practice Guideline — Psychological Assessment and Testing:

Selecting and Aministering Tests

Currency

Psychologists should use the most current edition of the test and norms unless there is compelling rationale to use a previous edition. Generally, it is expected that psychologists will adopt and use the current edition of a test for clinical use within one year of the release date. It would generallly be considered poor practice to use tests beyond two years after a new edition has been available…

In Decoding the Ethics Code: A Practical Guide for Psychologists, Section titled “Use of Obsolete Tests”, Fisher (2022) wrote the following when discussing APA Ethical Principles of Psychologists and Code of Conduct 9.08:

Test developers often construct new versions of a test to reflect significan (a) advances in the theoretical constructs underlying the psychological characteristic assesed; (b) transformation in cultural, educational, linguistic, or societal influences that challenge the extent to which current test items validly reflect content domains; or (c) changes in the demographci characteristics of the population to tested affecting the interpretations that can be drawn from standardized scores. Standard 9.08b prohibits psychologists from using outdated versions of tests for assessment or intervention decisions when interpretations drawn from the test are of questionable validity or otherwise not useful for the purposes of testing.
Fisher (2022)

In 2021, Ms. T filed complaints of unprofessional conduct against the three psychologists with the College of Alberta Psychologists. However, in September 2022, Dr. Troy Janzen, Complaints Director and Deputy Registrar, the College of Alberta Psychologists, dismissed all of Ms. T’s complaints on the grounds that Drs. Westcott, Mandel and Suffield’s conduct, including their use of outdated, obsolete, and irrelevant data sets to base their opinions about Ms. T, was at least minimally competent conduct.

Ms. T appealed the dismissals of her complaints to the College of Alberta Psychologists Complaints Review Committee (CRC) and the CRC review is pending.

In June 2023, Uttl (2023) gave presentations and led an ethics stream discussion session at the Canadian Psychological Association meeting in Toronto, ON, titled “Pronouncing opinions about clients based on obsolete data sets: Minimally competent practice, unprofessional conduct, and/or malpractice? The lively discussion continued well beyond the allocated time through most of the lunch break. In contrast to Dr. Janzen’s view, in participants’ opinions, a direct comparison of Ms. T’s WAIS-IV CDN (Wechsler, 2008) FSIQ scores to the mean IQ or GCT scores of some unknown, non-representative samples tested decades ago with different, non-equivalent tests amounted to no less than malpractice.

Drs. Westcott, Mandel, and Suffield’s letters to the College of Alberta Psychologists and/or expert reports featuring their use of outdated, obsolete, and irrelevant tests to make disparaging claims about Ms. T’s IQ, intelligence, and/or cognitive abilities are provided here for readers’ perusal:

References

Gottfredson, L. S. (1997). Why g matters: The complexity of everyday life. Intelligence, 24(1), 79–132. https://doi.org/10.1016/S0160-2896(97)90014-3

Gottfredson, L. S. (1998). The general intelligence factor. Scientific American, 9, 24–29.

Gottfredson, L. S. (2003). Chapter 15—G, Jobs and Life. In H. Nyborg (Ed.), The Scientific Study of General Intelligence (pp. 293–342). Pergamon. https://doi.org/10.1016/B978-008043793-4/50053-2

Holdnack, J. A., & Weiss, L. G. (2013). Chapter 4—Demographic Adjustments to WAIS–IV/WMS–IV Norms. In J. A. Holdnack, L. W. Drozdick, L. G. Weiss, & G. L. Iverson (Eds.), WAIS-IV, WMS-IV, and ACS (pp. 171–216). Academic Press. https://doi.org/10.1016/B978-0-12-386934-0.00004-3

Longman, R. S., Saklofske, D. H., & Fung, T. S. (2007). WAIS-III percentile scores by education and sex for U.S. and Canadian populations. Assessment, 14(4), 426–432. https://doi.org/10.1177/1073191107304114

Uttl, B. (2023, June). Pronouncing opinions about clients based on obsolete data sets: Minimally competent practice, unprofessional conduct, and/or malpractice? [Conversation Session/Ethics Stream]. Canadian Psychological Association, Toronto, ON. https://sd5bc.info/wp-content/uploads/2023/08/Uttl-CPA-2023-EthicsObsoleteTests.pdf

Using outdated, obsolete, and irrelevant test data to make disparaging statements about a client’s IQ, intelligence, and cognitive abilities: Minimally competent practice, unprofessional conduct, and/or malpractice?

Leave a Comment Cancel Reply