By Con Stough MAPS
Professor of Cognitive Neuroscience
Consultant to Channel 9’s National IQ Test 2002
National IQ Test 2002, aired on Channel 9 on Tuesday August 6, was the first show if its kind to be televised in Australia. It is unlikely to be eclipsed as the nation’s most watched show this year – measured at approximately 3.5 million viewers. In this article I will outline my involvement in National IQ Test 2002, from initial discussions with Channel 9 to my eventual appearance on the show. My purpose is to describe the process by which all of this occurred, and my motivations and dilemmas around working with a major television network on something as controversial as IQ.
No doubt some members of the APS will agree with my participation and some will not. Despite this, there are some interesting issues to consider and lessons to learn when dealing with the media, and I hope that I might further facilitate discussion about these issues. Overall, there are important questions for psychologists to consider about how we disseminate our knowledge and how we make our expertise understood and applied in the community.
Approach by Channel 9
In late May I was contacted by a team of producers at Channel 9. They had done some research to identify Australian psychologists active in intelligence research and had approached a number of‘experts’ to help them with their new show.
At a subsequent meeting with the producers in Melbourne, I discovered that the show involved administering an IQ test on television. The format for the program had been run successfully in several European countries: firstly in Holland and then in Germany, France and the United Kingdom. In fact some 500,000 people in the UK did the test online and approximately 12 million people watched the show.
Given my active research on intelligence and my frequent contributions to scientific journals on the subject, Channel 9 identified me as an Australian who could assist it to bring the UK experience to this country, and invited me to do so.
The ethical dilemma
My first reaction to the invitation was fear or at least strong apprehension. Although I was used to discussing my research with the media, and had a strong belief that academics should cooperate closely with the popular press to explain psychological research, this seemed to be quite different. This show was not about disseminating recent research from my research centre.
The construct of intelligence and the utility of intelligence tests are very important to me. So even the thought of administering an IQ test on television greatly concerned me, independent of whether I wanted to be involved.
I did not have the opportunity to discuss with Channel 9 whether administering an IQ test on national television and providing IQ scores to the Australian population was a good idea or not. Channel 9 had bought the rights to the show and there was no going back. Although they were keen to enlist my services as rapidly as possible, it was also clear to me that the show would go ahead with or without my involvement.
So what would my involvement entail? After some discussion my duties were to include advice on all aspects of the IQ testing for the show, the development of a multiple-choice adult IQ test to be administered via the television and the internet, and the development of a scaled-down multiple-choice IQ test for adolescents to be published in the Newscorp newspapers. Further, this task was to be achieved in three months!
So what was I to do? (1) Respectfully refuse any involvement and wish them well in their endeavours? (2) Suggest another colleague to assist them in their work? Or (3) Accept their offer and ensure that the work was undertaken at a high level of competence and that the numerous ethical considerations were treated and considered as carefully as possible.
On the surface it would appear that options (1) and (2) were the safest ones, with option (3) a great risk in terms of career and adverse publicity to the university and my centre’s program of research on intelligence. However, after some thought, I decided that options (1) and (2) were more likely to result in a poor test being constructed and its unethical treatment. A poorly constructed test would surely be ammunition for those who regarded IQ tests as unethical, and not valid or reliable. The great challenge in my own mind was to ensure that the test administered on national television reflected sound development and psychometric properties and that the feedback was handled as ethically as possible.
The latter point (how the feedback on test performance was handled) caused me the most loss of sleep. Although I was impressed by the insightful discussions at Channel 9 about the ethics of IQ test administration and feedback, the risk was that this could dissipate at any time up to and including the night of the television show.
IQ question development
In the end I decided that in order to develop a valid IQ test within three months, a number of steps were required in which I would need the strong support of my staff and postgraduate students at the Swinburne Centre for Neuropsychopharmacolo-gy, as well as the considerable resources of Channel 9. At the very least, several stages were required in the development of the Adult IQ test.
First, the test had to reflect my scientific view on how best to measure intelligence. This view ultimately resulted in the format of the actual test that was administered on the show. Thus the conceptualisation of the test and the different dimensions that comprised the test were to be my domain and not influenced by Channel 9.
I had in my possession the Dutch and UK IQ tests and regarded those to be very limited assessments of intelligence. I needed to implement some more modern methods of assessing intelligence. Interestingly, the previous version of the IQ test did not assess the application of general intelligence in terms of application to our community. I’ve always thought that it is one thing to have excellent arithmetic skills but another to show intelligence in making social decisions. This aspect of intelligence would have to be implemented in any IQ test created within my research centre.
Second, Channel 9 needed to provide a large sample of people (at least 1000) to further develop the numerous items that were constructed.
Third, Channel 9 needed to assist in recruiting additional participants to help equilibrate our test with, in my view, the gold standard for IQ testing: the Wechsler Adult Intelligence Scale III (WAIS III). This third stage was to help in establishing the norms for the test.
After Channel 9 willingly agreed to all of these requirements, I then had two important steps to complete before I could agree to my own participation, as well as that of my staff, postgraduate students and university.
First I needed to discuss my involvement with my Vice-Chancellor (VC) who was also trained in psychology. At this stage I was quite prepared for the VC to warn against any participation – there were significant risks for my career and for the university if the development of the test was poor, or if the test-performance feedback was unethical. However the VC was very positive and assigned a staff member from the media unit to provide some advice about handling Channel 9.
The second step was to enlist the support and help of the staff and students at the centre. After some discussion we decided that this would be a fun project and that we were equipped to complete the development and validation of the test. None of us were truly prepared for the enormity of the task, which required eight weeks of item development across many different and diverse cognitive abilities, and reliability and validity analyses. Certainly none of us would claim that it was the best test we had ever developed, but given the time constraints, we were satisfied with the end result: the dimensions were internally consistent; a general factor accounted for more than half of the variance despite the test comprising many different abilities from spatial, arithmetic, social, language, memory domains etc; and the test showed high correlations with the WAIS III.
Other factors I considered in agreeing to participate in the project included the opportunity to further educate the public about IQ testing and the construct of intelligence.
If we are to be the ‘clever country’, then surely we need to value the construct of intelligence. In Australia we value the contribution of our sporting heroes more than the contribution of intelligent individuals. If we are to stop the ‘brain drain’ to other countries, then we need to further invest in education, and in research and development. I believe there is a strong relationship between the way we undervalue the construct of intelligence and the fact that no Australian university is ranked in the top 100 universities in the world, or that as a nation we are ranked 18th in industrialised western nations in terms of Research and Development funding.
I had also been concerned about psychologists’ lack of public profile in Australia and the profile of the APS. Participating in National IQ Test 2002 allowed me to bring a psychological construct and the APS into the public gaze. Before the show I met with several members of the APS to discuss whether they would assist with the show. I suggested that we use the APS telephone number and logo to refer members of the public who were interested in further discussing their test score, or who wanted to have their IQ administered by a psychologist with a better test in a more controlled environment.
After listening to the genuine (and very valid) concerns of the APS, we all agreed that several disclaimers needed to be aired on the night. These were all discussed and agreed with Channel 9 before my meeting with the APS, but nevertheless helped reinforce in my own mind the importance of explaining the limitations of the test, administration of the test and the IQ scores generated from completing the test. Indeed it was essential that I also discussed that performance on the night would be influenced by a wide range of factors including anxiety, concentration, whether English was the participant’s first language, motivation etc.
On the night I was given scope to emphasise that an IQ test score was a limited predictor of life success and that other factors such as motivation, social intelligence, creativity, and even emotional intelligence, among others, may be as important or more important in many of our activities in everyday life.
Two days of rehearsals preceded the television show. The first half was recorded ‘live to tape’ and the second half live across Australia. No major difficulties occurred on the night and the show was successful beyond the expectations of Channel 9.
From my perspective the show improved the status of the construct, my professional society and profession, and my university. Since the show aired, there has been a lot of discussion about IQ and intelligence in general. There is no doubt the public has a fascination for many of the constructs that psychologists consider on a day-to-day basis.
In terms of achieving the outcomes that I thought were important from the start, I must give a lot of credit to the professionalism of the Channel 9 team in the development of the show.
Moreover, it would have been impossible without the hard work of the staff and students from my research centre at Swinburne (Jenny Lloyd, Karen Hansen, Stephanie Redman, Beata Silber, Cindi Hinch, Gilles Gignac, John Song and Cindy Van Rooy).
On the flip side of this was our endeavour to tread the thin line between entertainment and science. In the end I think there was good science around the show. But just as important, the show was fun and entertaining.
Many enquiries fielded at APS National Office
By Mick Symons, Manager, Member Services
THOSE who watched Test the Nation: IQ would know that the Referral Service phone number was mentioned towards the end of the program.
An extra staff member was rostered on for the Wednesday in anticipation of the public response. When reception staff checked the overnight messages, there were already 35 waiting.
Enquiries then came in great quantities, and covered many aspects from helping with the scoring process to suggestions about how the APS should run the event next time.
By the end of the week, more than 100 referral calls had been passed on to Members listed on the Referral Service.
By mid September, one month on, we are still receiving occasional enquiries.