II RNCE IAS abril 2015 copy.key
Transcrição
II RNCE IAS abril 2015 copy.key
II Reunião REDE NACIONAL DE CIÊNCIA PARA EDUCAÇÃO 27 de abril de 2015 Instituto Ayrton Senna, SP Competências Cognitivas no Século XXI - Medidas de Processos de Raciocínio, Conhecimento e Criatividade em sistemas de avaliação em larga escala Dr. Ricardo Primi, Universidade São Francisco, Itatiba, Brasil Instituto Ayrton Senna, São Paulo, Brasil EduLab21 Centro de Conhecimento do IAS Março 2015 Objetivos • Apresentar uma breve revisão sobre os modelos sobre inteligência (“estado da arte”) • Relacionar as competências do século XXI com os modelos mais recentes de inteligência (CHC) • Refletir e advogar a importância desse modelo para mapear o que avaliamos atualmente nos sistemas em larga escala e propor futuros aprimoramentos Competências do Século XXI www.p21.org Aprendizagem e inovação Criatividade Pensamento crítico e solução de problemas Comunicação e colaboração Habilidades para carreira e para vida Flexibilidade Adaptação à mudança Gerenciar objetivos e tempo Trabalhar independentemente Aprendizagem com auto-direção Interagir efetivamente com os outros Trabalhar efetivamente com a diversidade Liderança Responsabilidade para com os outros Competências do Século XXI Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century EDUCATION FOR LIFE AND WORK Developing Transferable Knowledge and Skills in the 21st Century Committee on Defining Deeper Learning and 21st Century Skills James W. Pellegrino and Margaret L. Hilton, Editors Copyright © National Academy of Sciences. All rights reserved. Board on Testing and Assessment and Board on Science Education Cluster Division of Behavioral and Social Sciences and Education Cognitive Processes and Strategies Copyright © National Academy of Sciences. All rights reserved. COGNITIVE COMPETENCIES Knowledge Creativity Terms Used for 21st Century Skills Critical thinking, problem solving, analysis, reasoning/argumentation, interpretation, decision making, adaptive learning, executive function 32 TABLETABLE 2-2 Clusters of 21st Century Competencies 2–2 Clusters of 21st Century Competencies O*NET Skills System skills, process skills, complex problemsolving skills Main Ability/ Personality Factor Main ability factor: fluid intelligence (Gf) Information literacy (research Content skills using evidence and recognizing bias in sources); information and communications technology literacy; oral and written communication; active listening Main ability factor: crystallized intelligence (Gc) Creativity, innovation Main ability factor: general retrieval ability (Gr) Complex problemsolving skills (idea generation) Cluster Intellectual Openness INTRAPERSONAL COMPETENCIES Work Ethic/ Conscientiousness Positive Core SelfEvaluation Terms Used for 21st Century Skills O*NET Skills Main Ability/ Personality Factor Flexibility, adaptability, artistic [none] and cultural appreciation, personal and social responsibility (including cultural awareness and competence), appreciation for diversity, adaptability, continuous learning, intellectual interest and curiosity Main personality factor: openness Initiative, self-direction, responsibility, perseverance, productivity, grit, Type 1 selfregulation (metacognitive skills, including forethought, performance, and selfreflection), professionalism/ ethics, integrity, citizenship, career orientation [none] Main personality factor: conscientiousness Type 2 self-regulation (self[none] monitoring, self-evaluation, selfreinforcement), physical and psychological health Main personality factor: emotional stability (opposite end of the continuum from neuroticism) 33 continued Cluster INTERPERSONAL COMPETENCIES SOURCE: Created by committee. 34 TABLE 2-2 Continued Terms Used for 21st Century Skills O*NET Skills Main Ability/ Personality Factor Competências do Século XXI • "do século XXI" ou "mais valorizadas em nosso tempo”? • capacidades já conhecidas • Qual o “estado da arte" do campo de estudo da inteligência? Qual o conjunto de evidências que baseiam os sistemas de avaliação em larga escala ? • Quais medidas compõem os sistemas de avaliação em larga escala ? Todos os fatores cognitivos relevantes são cobertos ? • Abordagens da Inteligência • Educação foco no conhecimento • Psicologia foco nos processos mentais Breve histórico da evolução das teorias sobre a inteligência Fator g de Charles Spearman • 1904 General intelligence objectively determined and measured • "positive main fold”. Invenção da análise fatorial. Modelo bifatorial. “energia mental”. neogenese. • (a) edução de relações capacidade maior ou menor de estabelecer relações entre duas ou mais idéias; • (b) edução de correlatos capacidade maior ou menor que as pessoas demonstram de criar novas idéias a partir de uma idéia e uma relação. • (c) apreensão das experiências capacidade à rapidez e à acuidade com que as pessoas percebem os estímulos, e bem como aos processos de; Quase 100 anos depois: Duncan, J., Seitz, R. J., Kolodny, J., Bor, D., Herzog, H., Ahmed, A., ... & Emslie, H. (2000). A neural basis for general intelligence. Science, 289(5478), 457-460. Teoria do Investimento de R. Cattell • Inteligência Fluida Gf: capacidade de raciocínio “uso de operações mentais controladas deliberadamente para resolver problemas novos... que não podem ser resolvidos automaticamente” .. “ operações mentais como: fazer inferências, formar conceitos, classificar, gerar e testar hipóteses, identificar relações, compreender implicações, resolver problemas, extrapolar e transformar informações” (MacGrew & Evans, 2004; Kane & Gray, 2005) • Inteligência Cristalizada Gc Riqueza (profundidade e amplitude) do estoque dos conhecimentos adquiridos Variáveis Cognitivas e Não Cognitivas na Escola: Modelo de J. B. Carroll (1963) Quantidade de tempo engajado na tarefa Aprendizagem efetiva = Quantidade de tempo necessário Quantidade de tempo engajado na tarefa = Oportunidade + Motivação/Per sistência Quantidade de tempo necessário 1 = Capacidade prévia + Qualidade do ensino 2014 *AMAR25DOM19* MATEMÁTICA E SUASimportados TECNOLOGIAS Uma loja vende produtos e QUESTÃO 137 nacionais entre vestidos camisas e casacos. Questões de 136 a 180 Uma empresa que organiza eventos de formatura confecciona canudos de diplomas a partir de folhas de papel TXDGUDGDV 3DUD TXH WRGRV RV FDQXGRV ¿TXHP LGrQWLFRV cada folha é enrolada em torno de um cilindro de madeira de diâmetro d em centímetros, sem folga, dando-se 5 voltas FRPSOHWDVHPWRUQRGHWDOFLOLQGUR$R¿QDODPDUUDVHXP cordão no meio do diploma, bem ajustado, para que não RFRUUDRGHVHQURODPHQWRFRPRLOXVWUDGRQD¿JXUD Alguns vestidos e todos os casacos fazem parte QUESTÃO 136importados. Não há produto dos produtos importado em uma tamanho grande. A Figuradisponível 1 representa gravura retangular com qual dentree os enunciados não 8 Assinale m de comprimento 6 mfatos de altura. poderia ser verdadeiro: 6 metros A. Carla experimenta uma camisa nacional. B. Luciana está comprando um casaco pequeno C. Alberto pegou um casaco grande D. Adriana experimenta um vestido pequeno. Em seguida, retira-se o cilindro de madeira do meio GR SDSHO HQURODGR ¿QDOL]DQGR D FRQIHFomR GR GLSORPD Considere que a espessura da folha de papel original seja desprezível. Qual é a medida, em centímetros, do lado da folha de papel usado na confecção do diploma? 8 metros Figura 1 Deseja-se reproduzi-la numa folha de papel retangular com 42 cm de comprimento e 30 cm de altura, deixando livres 3 cm em cada margem, conforme a Figura 2. 3 cm Folha de papel 3 cm A Sd B 2 Sd C 4 Sd D 5 Sd E 10 Sd QUESTÃO 138 o posso levar ui de novo cantando ui de novo xaxando i de novo mostrando eve xaxar E treinar o país em segurança digital. QUESTÃO 102 orena linda chita mais bonita u lugar Maria, chama Luzia Zabé, chama Raque tou aqui com alegria ROS, A. Óia eu aqui de novo. Disponível em: www.luizluagonzaga.mus.br. Acesso em: 5 maio 2013 (fragmento). a canção de Antônio de Barros manifesta do repertório linguístico e cultural do Brasil. ue singulariza uma forma característica do ar regional é: um desaforo”. e eu tou aqui com alegria”. ostrar pr’esses cabras”. hama Maria, chama Luzia”. á morena linda, vestida de chita”. O 101 a escala de 0 a 10, o Brasil está entre 3 e 4 no gurança da informação. “Estamos começando ara o problema. Nessa história de espionagem , temos muita lição a fazer. Falta consciência al e um longo aprendizado. A sociedade caiu u que é uma coisa que nos afeta”, diz S.P., em segurança da informação. Para ele, devem ecidos canais de denúncia para esse tipo de De acordo com o conselheiro do Comitê Gestor (CGI), o Brasil tem condições de desenvolver própria para garantir a segurança dos dados anto do governo quanto da população. “Há a de conhecimento dentro das universidades mpresas inovadoras que podem contribuir medidas para que possamos mudar isso WILL. Disponível em: www.willtirando.com.br. Acesso em: 7 nov. 2013. Opportunity é o nome de um veículo explorador que aterrissou em Marte com a missão de enviar informações à Terra. A charge apresenta uma crítica ao(à) A B C D E gasto exagerado com o envio de robôs a outros planetas. exploração indiscriminada de outros planetas. circulação digital excessiva de autorretratos. vulgarização das descobertas espaciais. mecanização das atividades humanas. Contents lists available at ScienceDirect Learning and Individual Differences journal homepage: www.elsevier.com/locate/lindif The relationship between intelligence and academic achievement throughout middle school: The role of students' prior academic performance Diana Lopes Soares a,⁎, Gina C. Lemos a, Ricardo Primi b, Leandro S. Almeida a a b Centro de Investigação em Educação, Instituto de Educação, Universidade do Minho, Portugal Departamento de Psicologia, Universidade de S. Francisco, Brazil a r t i c l e i n f o a b s t r a c t Article history: Received 4 April 2014 Received in revised form 27 January 2015 Accepted 18 February 2015 Available online xxxx Keywords: Intelligence Academic achievement Mediation analysis Investment Theory Gf–Gc The association between intelligence and academic achievement is well established. However, how this relationship changes throughout schooling remains undefined. In this 3-year longitudinal study, 284 Portuguese middle school students completed three reasoning subtests (abstract, numerical, and verbal) by the end of 7th grade (intelligence), and their academic grades were collected at the same time (prior academic achievement, AA7) and by the end of 9th grade (final academic achievement, AA9). The main findings show that i) when intelligence and AA7 are analyzed as two independent predictors of AA9, AA9 is best predicted by intelligence when considering the mediation effect of AA7, and ii) the inclusion of AA7 in the pathway between intelligence and AA9 produces a considerable increase in the predictive validity of intelligence. Implications for cognitive assessment and psychological practice are emphasized based on this Gf–Gc relationship. © 2015 Elsevier Inc. All rights reserved. D.L. Soares et al. / Learning and Individual Differences xxx (2015) xxx–xxx 5 1. Introduction Primi, Ferrão, & Almeida, 2010; Rohde & Thompson, 2007; Watkins, Lei, & Canivez, 2007; Weber, Lu, Shi, & Spinath, 2013). General intelligence, also named fluid intelligence – Gf (Cattell, An extensive body of research has been developed in order to 1971), is usually measured by administering tests of inductive and understand the relationship between intelligence and achievement deductive reasoning, which are assumed to reflect the ability to think, in different life domains, such as job performance (Gottfredson, solve problems, make inferences, identify relations, and transform 2002; Schmidt & Hunter, 2004), health outcomes (Der, Batty, & information in a significant way (Lemos, Almeida, & Colom, 2011; Deary, 2009), or even wealth (Zagorsky, 2007). Particularly in educaNickerson, 2011). Longitudinal growth modeling attests that Gf tional settings, intelligence plays a crucial role in learning and predicts, not only the initial level of math achievement, but also, the academic performance. Several studies show high correlation rate of change in learning and achievement (Primi et al., 2010). indices between them, ranging from .50 to .70 (Lynn & Vanhanen, In turn, academic achievement is usually measured by administering 2012). For instance, Mackintosh (1998) revealed a .50 correlation tests to assess knowledge that is formally taught in schools. As a broad between 11-year-olds' intelligence scores and later educational concept, achievement could be associated with crystallized intelliachievement at the age of 16. In addition, in a 5-year longitudinal gence – Gc, which is defined as the “depth and breadth of knowledge study with 70,000 children, Deary, Strand, Smith, and Fernandes that are valued by one's culture” (Schneider & McGrew, 2012, p. 122). (2007) found a .81 correlation between intelligence at the age of 11 Accordingly, Schneider (2013) claims that intelligence is and educational achievement at the age of 16 in 25 academic related to potential, and achievement to the execution of potential. subjects. Other studies also identify intelligence as a relevant predicAlthough considered as separate abilities, both are viewed as two tor of academic achievement (Colom & Flores-Mendoza, 2007; aspects of the g factor according to Cattell's investment theory Karbach, Gottschling, Spengler, Hegewald, & Spinath, 2013; Laidra, In light of Cattell'sINTEL model,= fluid intelligence is one of the & Allik, 2007; Lemos, Abad, achievement, Almeida, & Colom, 2013; Fig. 2. Model of indirect effect Pullmann, from intelligence to final academic through prior (1971). academic achievement. intelligence; AA7 = academic achievement in 7th grade main causes achievement, since NR more to learn predicts P7 = Portuguese score in 7th (prior grade); AA9 = academic achievement in 9th grade (final grade); AR = abstract reasoning; VR =ofverbal reasoning; =capacity numerical reasoning; more efficient and rapid learning. This potential is invested in grade; E7 = English score in 7th grade; Mat7 = mathematics score in 7th grade; SC7 = sciences score in 7th grade; P9 = Portuguese score in 9th grade; E9 = English score in 9th experiences, and is transformed into knowledge, that is, crystal⁎ Corresponding author at: Centro de Investigação em Educação, Universidade do grade; Mat9 = mathematics score in 9th grade; SC9 = sciences score in 9th grade. lized intelligence. In the process of transformation of potential Minho, Campus de Gualtar, 4710-057 Braga, Portugal. Tel.: +351 927433844. (Gf) into fulfilled potential (Gc), other factors play a role, such as E-mail address: [email protected] (D.L. Soares). http://dx.doi.org/10.1016/j.lindif.2015.02.005 only prior level of academic achievement is a direct predictor of subsedifferent analytic approaches. In model 1, intelligence and prior 1041-6080/© 2015 Elsevier Inc. All rights reserved. quent learning, explaining about 70% of its variance. However, when the academic achievement were considered together as a direct predictor Please cite this article as: Soares, D.L., et al., The relationship between intelligence and academic achievement throughout middle school: The role indirect effect of intelligence on final academic achievement through of final academic achievement. In model 2, both variables were tested of students' prior academic..., Learning and Individual Differences (2015), http://dx.doi.org/10.1016/j.lindif.2015.02.005 prior level of academic achievement (model 2) is tested, intelligence as autonomous predictors of final academic achievement, and direct only has an indirect effect on final academic grades. When including and indirect effects were also tested. b·sicas 47 Estudos deHabilidades Psicologia 2002, 7(1), 47-55 Habilidades b·sicas e desempenho acadÍmico em universit·rios ingressantes Ricardo Primi Ac·cia A. Angeli dos Santos. Claudette Medeiros Vendramini Universidade S„o Francisco Resumo Recentes estudos sobre o desenvolvimento cognitivo adulto referem-se ‡ distinÁ„o entre inteligÍncia fluida como a capacidade geral de relacionar idÈias complexas, formar conceitos abstratos e derivar implicaÁıes lÛgicas a partir de regras gerais e inteligÍncia cristalizada como a capacidade de derivar conhecimento a partir de esquemas organizados de informaÁıes sobre disciplinas especÌficas. Para verificar a possÌvel relaÁ„o entre a habilidade cognitiva requerida e a ·rea de conhecimento, este estudo foi proposto com o objetivo de investigar as correlaÁıes entre medidas de inteligÍncia fluida e cristalizada com desempenho acadÍmico em 960 alunos ingressantes dos cursos de Medicina, Odontologia, Engenharia Civil, Matem·tica, Psicologia, Pedagogia, Letras e AdministraÁ„o. As correlaÁıes encontradas indicam que o desempenho acadÍmico est· associado a diferentes perfis de habilidades cognitivas. 0,22 S ó c . G e o g r. 0,75 N a t. M a t. 0,82 Palavras-chave: Habilidades b·sicas, InteligÍncia fluida e cristalizada, AvaliaÁ„o educacional, Teorias de inteligÍncia. 0,88 0,50 Gc 0,43 Abstract in freshmen students 0 , 0 9 Basic (t= abilities 2 , 5 5 and , pscholastic < 0 , 0achievement 5) Recent studies of adult cognitive development have distinguished two core abilities: fluid intelligence as a general capacity to infer complex relationships, abstract concepts, and to deduce logical implications from general rules; and crystallized intelligence as a capacity to solve problems by using organized knowledge schemes from specific disciplines. In order to investigate a possible relationship between cognitive abilities and area of study, this research studied the correlation between fluid and crystallized intelligence measures and academic achievement among 960 freshmen students of eight areas of study: Medicine, Dentistry, Psychology, Business, Engineering, Mathematics, Education, and Literature. The results have indicated that the academic achievement is correlated with different abilities profile. L in g . V e rb . RM G 0,52 0,57 0,77 RLD RI CL 0,69 0,65 0,48 RM 0,49 DP 0,45 -‐ 1 , 0 4 0 , 4 1 (t= 9 , 3 9 , p < 0 , 0 0 1 ) Key words: Basic abilities, Fluid and crystallized intelligence, Educational assessment, Intelligence theories. Gf C om as transformaÁıes pelas quais a sociedade tem passado, cada vez mais tem sido apontado que ser· necess·rio ao profissional das prÛximas geraÁıes, n„o apenas o domÌnio de conhecimentos especÌficos, mas a capacidade de se adaptar rapidamente e assimilar novas informaÁıes de um mundo em constante transformaÁ„o. Isso vem se tornando um prÈ-requisito b·sico para o perfil do profissional do novo milÍnio. Portanto, a universidade dever·, cada vez mais, produzir estratÈgias que privilegiem, n„o sÛ a aprendizagem de conte˙dos, mas tambÈm a aprendizagem de estratÈgias de adaptaÁ„o a situaÁıes novas. Essa capacidade, definida pela psicologia como ìinteligÍncia geralî, foi alvo de v·rias pesquisas desde o inÌcio deste sÈculo (Almeida, 1988; Sternberg, 1981). Os estudos psicomÈtricos, que investigaram como as habilidades humanas se estruturam, chamaram-na de fator g, enquanto em estudos mais recentes desenvolvidos pela psicologia cognitiva, v·rios dos processos que poderiam ser classifica- Meta análise de John B. Carroll • Carroll (1993): marco histórico nas teorias psicométricas • Revisou os principais estudos psicométricos dos últimos 60 anos (1500 referências) • Refez a análise fatorial de 461 matrizes de correlação destes estudos • Apresentou um modelo integrado de Três Camadas • Modelo foi integrado na Teoria Cattell-Horn-Carroll (CHC) organizando a inteligência em Camadas 1. Fator g, II 10 Fatores Amplos e III 70 fatores específicos Modelo Cattell-Horn-Carroll Contemporary psychometric research has converged on the Cattell-Horn-Carroll (CHC) theory of cognitive abilities as the consensus working taxonomy of human intelligence McGrew, K. (2009). Editorial: CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research, Intelligence, 37, 1-10. McGrew (2009), McGrew & Schneider (2012, 2013) Stratum III (general) g Gf Carroll and Cattell-Horn Broad Ability Correspondence (vertically-aligned ovals represent similar broad domains) A. Carroll Three-Stratum Model Gc Gy Gv Stratum II (broad) Gu Gr Gs Gt 80+ Stratum I (narrow) abilities have been identified under the Stratum II broad abilities. They are not listed here due to space limitations (see Table 1) B. Cattell-Horn Extended Gf-Gc Model Gf Gc SAR Gsm Gv Ga TSR Glm Gs CDS Grw Gq C. Cattell-Horn-Carroll (CHC) Integrated Model D. Tentatively identified Stratum II (broad) domains g Gf Gc Gsm Gv Ga Glr Gs Gt Grw Gq Gkn Gh (Missing g-to-broad ability arrows acknowledges that Carroll and Cattell-Horn disagreed on the validity of the general factor) CHC Broad (Stratum II) Ability Domains Gf Gc Gsm Gv Ga Glr Gs Gt Grw Gq Fluid reasoning Comprehension-knowledge Short-term memory Visual processing Auditory processing Long-term storage and retrieval Cognitive processing speed Decision and reaction speed Reading and writing Quantitative knowledge Gkn Gh Gk Go Gp Gps General (domain-specific) knowledge Tactile abilities Kinesthetic abilities Olfactory abilities Psychomotor abilities Psychomotor speed (see Table 1 for definitions) Gk Go Gp Gps Grw Gq Gc Gkn Acquired Knowledge Go Ga Gk Gh Gv Gp Sensory Motor Sensory-Motor Domain-Specific Abilities (From Schneider & McGrew, 2012) Gf Gps Gsm Glr Memory Conceptual Grouping Functional Grouping Gs Gt General Speed Parameters of Cognitive Efficiency Domain-Independent General Capacities Cattell-Horn-Carroll Theory of Cognitive Abilities Controlled Motor Perception Attention Knowledge Fluid Gp Psychomotor Gk Gv Reasoning Gf Ga Gsm Gh Go Short-Term Abilities Gq Grw Gc Gkn Memory Gps Gt Gs Glr Psychomotor Speed Attentional Learning Efficiency Speed of Perception Fluency & Retrieval Fluency Speed Cattell-Horn-Carroll Theory of Cognitive Abilities Controlled Motor Perception Attention Knowledge Fluid Gp Psychomotor Gk Gv Reasoning Gf Ga Gsm Gh Go Short-Term Abilities Gq Grw Gc Gkn Memory Gps Gt Gs Glr Psychomotor Speed Attentional Learning Efficiency Speed of Perception Fluency & Retrieval Fluency Speed This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Glr: Fluência de Produção Psychology of Aesthetics, Creativity, and the Arts 2014, Vol. 8, No. 4, 000 © 2014 American Psychological Association 1931-3896/14/$12.00 http://dx.doi.org/10.1037/a0038055 Divergent Productions of Metaphors: Combining Many-Facet Rasch Measurement and Cognitive Psychology in the Assessment of Creativity Ricardo Primi University of São Francisco This article presents a new method for the assessment of creativity in tasks such as “The camel is ________ of the desert.” More specifically, the study uses Tourangeau and Sternberg’s (1981) domain interaction model to produce an objective system for scoring metaphors produced by raters and the many-facet Rasch measurement to model the rating scale structure of the scoring points, item difficulty, and rater severity analysis, thus making it possible to have equated latent scores for subjects, regardless of rater severity. This study also investigates 4 aspects of the method: reliability, correlation between quality and quantity, criterion validity, and correlation with fluid intelligence. The database analyzed in this study consists of 12,418 responses to 9 items that were given by 975 persons. Two to 10 raters scored the quality and flexibility of each metaphor on a 4-point scale. Raters were counterbalanced in a judge-linking network to permit the equating of different “test forms” implied in combinations of raters. The reliability of subjects’ latent quality scores was .88, and the correlation between quality and quantity was low (r ! ".14), thus showing the desired separation between the 2 parameters established for the task scores. The latent score on the test was significantly associated with the profession that requires idea production (r ! .19), and the latent scores for the correlation between creativity and fluid intelligence were high, # ! .51, even after controlling for crystalized intelligence (r ! .47). Mechanisms of fluid intelligence, executive function, and creativity are discussed. Keywords: metaphor production, intelligence, creativity, item response theory, Rasch measurement duced teste for prompts such as “The camel is ________ of the desert” Nesse gostaríamos que você inventasse metáforas que complete as frases apresentadas. Veja o exemplo abaixo: (example: boat). This study applies principles of cognitive psychology The assessment of intelligence has evolved immensely over the last century (Schneider & McGrew, 2012), and there is now consensus taxonomy regarding the classification of cognitive factors and methods of assessment that intelligence tests employ. The assessment of creativity, conversely, has not reached such a level of development. Recent works discuss methods, propose new ways to assess, and make a call for the development of different methods to evaluate creativity (Silvia et al., 2008). This article is a response to this call and presents a new method for assessing the quality of metaphors pro- to produce ratings that are potentially more objective and tied to the underlying associative process that is believed to be the foundation of idea production. It also illustrates the application of the psychometric method based on the Rasch model, which is useful when several judges rate different ideas generated by different people and thereby provide parameters for simultaneously measuring creativity, item difficulty, and rater severity. Ricardo Primi, Graduate Program of Psychology at Univesity of São Francisco. This article was financed by the by the Brazilian National Research Council (CNPq). The author acknowledges the contributions of Débora Pereira de Barros, Fabiano Koich Miguel, Fernando Pessotto, Gleiber Couto, Lucas de Francisco Carvalho Maria Beatriz Zanarella Cruz, Marjorie Cristina Rocha da Silva, Marta Petrini, Monalisa Muniz, Priscilla Rodrigues Santana, Sanyo Drummond Pires, Tatiana de Cássia Nakano, and Tatiana Freitas da Cunha, who were graduate students working under the supervision of the author and who contributed in data collection and as raters of the metaphors. Correspondence concerning this article should be addressed to Ricardo Primi, Universidade São Francisco, Laboratório de Avaliação Psicológica e Educacional [Laboratory of Psychological and Educational Assessment], Rua Alexandre Rodrigues Barbosa, 45, CEP 13251-900, Itatiba, São Paulo, Brazil. E-mail: [email protected] One mode of creativity assessment is based on divergent production tasks that ask people to produce as many ideas as they can from a prompt. Examples of prompts include the following: “List words that start with ‘s’” (word fluency), “List opposites of the word ‘true’” (associational fluency), and “Create titles for a story plot” (ideational fluency). These tasks are traced back to Guilford’s (1950, 1956, 1957) structure of intellect model, wherein he defined four factors—fluency (number of ideas), flexibility (shift in response category or approach to a problem), originality (unconventionality, remote associations, and response cleverness), and elaboration (amount of details in a response)—that are used in tests of creativity. These factors are measured using the widely known Torrance Tests of Creative Thinking (TTCT; Kim, 2006a, 2011; Plucker, 1999). Some psychometric problems with divergent thinking tests have been noted (Houtz & Krug, 1995; Plucker & Runco, 1998; Silvia O camelo é o/a _________________ do deserto Metáfora 1) Assessment barco Creativity Explicação No mar o barco é um meio de transporte que anda balançando como o camelo no deserto 2) moto Porque a moto é um transporte para uma ou duas pessoas e anda com pouco combustível como o camelo no deserto que precisa de pouca água 3) lesma Porque anda devagar, marcando o chão e rebolando como camelo 4) Barrichello Porque quando não está parado está andando devagar 1 Cattell-Horn-Carroll Theory of Cognitive Abilities Controlled Motor Perception Attention Knowledge Fluid Gp Psychomotor Gk Gv Reasoning Gf Ga Gsm Gh Go Short-Term Abilities Gq Grw Gc Gkn Memory Gps Gt Gs Glr Psychomotor Speed Attentional Learning Efficiency Speed of Perception Fluency & Retrieval Fluency Speed This document is copyrighted by the American Psychological Ass This article is intended solely for the personal use of the individual u This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Gf: Raciocínio, Memória de Trabalho e Funções executivas Psychological Assessment 2014, Vol. 26, No. 2, 000 © 2014 American Psychological Association 1040-3590/14/$12.00 http://dx.doi.org/10.1037/a0036712 Developing a Fluid Intelligence Scale Through a Combination of Rasch Modeling and Cognitive Psychology Ricardo Primi University São Francisco Ability testing has been criticized because understanding of the construct being assessed is incomplete and because the testing has not yet been satisfactorily improved in accordance with new knowledge from cognitive psychology. This article contributes to the solution of this problem through the application of item response theory and Susan Embretson’s cognitive design system for test development in the development of a fluid intelligence scale. This study is based on findings from cognitive psychology; instead of focusing on the development of a test, it focuses on the definition of a variable for the creation of a criterion-referenced measure for fluid intelligence. A geometric matrix item bank with 26 items was analyzed with data from 2,797 undergraduate students. The main result was a criterion-referenced scale that was based on information from item features that were linked to cognitive components, such as storage capacity, goal management, and abstraction; this information was used to create the descriptions of selected levels of a fluid intelligence scale. The scale proposed that the levels of fluid intelligence range from the ability to solve problems containing a limited number of bits of information with obvious relationships through the ability to solve problems that involve abstract relationships under conditions that are confounded with an information overload and distraction by mixed noise. This scale can be employed in future research to provide interpretations for the measurements of the cognitive processes mastered and the types of difficulty experienced by examinees. Keywords: inductive reasoning, fluid intelligence, Rasch measurement, matrix reasoning There has been significant development in cognitive psychology and the psychometrics of intelligence testing over the last few decades (Cornoldi, 2006; Deary, 2001; Whitely, 1980; Whitely & Schneider, 1981). Cognitive task analysis of items that are commonly used in intelligence tests provides a better understanding of how people represent and process information, which in turn improves a test’s score for construct validity (Sternberg, 1981). Psychometric model-based methods, including item response theory (IRT), provide ways to construct scales that show links between test scores and the underlying construct that provide substantial additional interpretations (Embretson, 2006; Wilson, 2005). These methods have recently evolved into cognitive diagnostic assessment models such as that of Tatsuoka (2009). Despite these developments, test construction procedure has not yet satisfactorily incorporated these new methods (Embretson, 1994). Thus, the expectation for the future generation of tests is that cognitive psychology should play an important role in construct representation. By providing a rich theoretical basis for the creation of purified tasks, it should lead to the development of instruments with better theoretical grounding. This article presents an illustration of these methods as they apply to the development of a fluid reasoning test. First, a review of the psychometric and cognitive neuroscience definitions of fluid intelligence is presented. A brief review of how fluid intelligence is measured follows, with emphasis on the justification for new tests based on modern methodology. Finally, an empirical study of test development is presented. Nature of Fluid Intelligence This article was developed as part of a larger project, called Development of a Computerized Componential Fluid Intelligence Test, which was financed by the Foundation for the Support of Research in the State of São Paulo (FAPESP, Process 2000/05913-4), by the Brazilian National Research Council (CNPq), and by the University of São Francisco. The author acknowledges the contributions of Acácia A. Angeli dos Santos, Claudette Medeiros Vendramini, Fernanda de Oliveira Soares Taxa, Maria de Fátima Lukjanenko, Franz A. Müller, Isabel Sampaio, Fatima Keiko Kuse, and Cíntia Heloína Bueno for their valuable comments on drafts of this article and assistance in collection of the data. Correspondence concerning this article should be addressed to Ricardo Primi, Universidade São Francisco, Laboratório de Avaliação Psicológica e Educacional, Rua Alexandre Rodrigues Barbosa, 45, CEP 13251-900, According to Schneider and McGrew (2012), fluid reasoning (Gf) refers to Figure 1. Examples of fluid intelligence items used in the present study and a summary of sources of complexity for matrix items and their link with fluid intelligence capacities. the deliberate but flexible control of attention to solve geometric novel “on the spot” problems that cannot be performed by relying exclusively on previously learned habits, schemas and scripts. Fluid reasoning is a multi-dimensional construct, but its parts are unified in their purpose: solving unfamiliar problems. Fluid reasoning is most evident in abstract reasoning that depends less on prior learning. However, it is also present in day-to-day problem solving. Fluid reasoning is typically employed in concert with background knowledge and automatized responses. (p. 111) Fluid intelligence is central to understanding the construct of are not confounded by the degree of mo results reported here. Discussion Psychological Bulletin 2005, Vol. 131, No. 1, 30 – 60 Copyright 2005 by the American Psychological Association 0033-2909/05/$12.00 DOI: 10.1037/0033-2909.131.1.30 Figure 1. Working Memory and Intelligence: The Same or Different Constructs? Phillip L. Ackerman, Margaret E. Beier, and Mary O. Boyle Georgia Institute of Technology Several investigators have claimed over the past decade that working memory (WM) and general intelligence (g) are identical, or nearly identical, constructs, from an individual-differences perspective. Although memory measures are commonly included in intelligence tests, and memory abilities are included in theories of intelligence, the identity between WM and intelligence has not been evaluated comprehensively. The authors conducted a meta-analysis of 86 samples that relate WM to intelligence. The average correlation between true-score estimates of WM and g is substantially less than unity (!ˆ ! .479). The authors also focus on the distinction between short-term memory and WM with respect to intelligence with a supplemental meta-analysis. The authors discuss how consideration of psychometric and theoretical perspectives better informs the discussion of WM–intelligence relations. Since the 1980s, with the major theoretical and empirical developments of the construct of working memory (WM; see, e.g., Baddeley, 1986; Richardson, 1996, for reviews) as distinct from rote or span memory (which is usually referred to as short-term memory [STM]), several investigators have asserted that WM and intellectual abilities are highly related or identical constructs. ThesePsychological assertions started with demonstrations that significant corBulletin 2005, Vol. 131, No. 1, 66 –71 relations were found between some measures of WM and measures of comprehension (Daneman & Carpenter, 1980), and later between WM and reasoning ability (Kyllonen & Christal, 1990), and other measures, such as the SAT (e.g., Turner & Engle, 1989). Recently, several investigators have claimed that WM and general intelligence (g; or general fluid intelligence, Gf) are essentially the same constructs. For example: A hierarchical factor analytic representation of g and first-order factors, based on a structural equation model of meta-analytically derived correlations among ability and working memory (WM) measures. the loading was .40, for Quantitative Reasoning the loading was .51, for Lexical Knowledge (Verbal) the loading was .37, and for Memory Span, the loading on g was .36. In the current SEM analysis, all of these loadings of the first-order factors on g were As we noted earlier, SEM with meta-a relations (i.e., where different correlation measures, different samples, and different is not optimal from a psychometric persp results with some degree of skepticism is less, neither SEM solution points to an in are isomorphic to one another. We were a which WM measures and ability measu single factor in favor of a model with separ though the estimated correlation between in this model was .50. The initial model provide a representation that is more conc abilities literature, given the presence o level cognition. The construct is distinguishable from STM and is at least related to, maybe isomorphic to, general fluid intelligence and executive attention. (Engle, 2002, pp. 21–22) No other cognitive factor— knowledge, speed, or learning ability— correlated with g after the working memory factor was partialed out. Thus, we have our answer to the question of what g is. It is working memory capacity. (Kyllonen, 2002, see also Kyllonen, 1996) Copyright 2005p.by433; the American Psychological Association 0033-2909/05/$12.00 DOI: 10.1037/0033-2909.131.1.66 However, the relationship between memory and intelligence appears to be much more complex than has been asserted by these investigators. Note that this position is not without its critics. For example, differential psychologists such as Deary (2000) and Kline (2000) have expressed substantial skepticism that WM and general intelligence are even closely linked. In this article, we evaluate these claims in the context of a meta-analysis of correlaJ. Kane Davidand Z. intellectual Hambrickability measures. In So central is the role of Michael WM capacity in individual differences in tions between WM measures Greensboro Michigan State University information University processing of thatNorth someCarolina cognitiveattheorists equate WM addition, it is important to note that intelligence theory and the capacity with g itself. (Jensen, 1998, p. 221) assessment of intelligence have both involved memory abilities the past 110 years. Although models of WM represent relaAndrew A. Conway Stauffer et al. (1996) found a correlation of " 0.995 between a factor R. over tively developments in the history of the science, an underUniversity of Illinois atrecent Chicago representing general intelligence (g) and a factor representing WM. standing of the construct space for individual differences in intel(Colom, Flores-Mendoza, & Rebollo, 2003, p. 34)1 ligence and WM benefits from a brief review of intelligence and My colleagues and I used a structural equation modeling analysis to memory ability research. Thus, we begin with a consideration of test this and the idea that theauthors construct byAckerman, WM-capacity The agreemeasured with P. L. M. E. Beier, and M.abilities O. Boyle from (2005)an thatintelligence working memory memory assessment perspective, tasks is closely associated with general . . . general WM- fluid intelligence (Gf) or reasoning ability. However, the capacity (WMC) fluid is not intelligence. isomorphic with followed by a review of research on memory abilities and intellicapacity tasks measure a construct important to WMC andfundamentally Gf/reasoning constructs arehighermore strongly associated than Ackerman et al. (2005) indicate, gence theory. we briefly review memory theory and the particularly when considering the outcomes of latent-variable studies.Next, The authors’ reanalysis of 14 such underlying for asserting thea strong overlap between WM and data sets from 10 published studies, representing more than 3,100 framework young-adult subjects, suggests A meta-analysis samples that report correlations correlation between WMC and Gf/reasoning factorsintelligence. (median r ! .72), indicating that of the86 WMC and Gf Phillip L. Ackerman, Margaret E. Beier, and Mary O. Boyle, School of constructs share approximately 50% of their variance. This comment also the measures authors’ “execbetween measures ofclarifies WM and of intellectual abilities is Psychology, Georgia Institute of Technology. utive attention” view of WMC, it demonstrates then that WMC has greater discriminant validity is than presented. A parallel set of analyses also provided for STM Margaret E. Beier is now atAckerman the Schooletof Riceand University. al.Psychology, (2005) implied, it suggests some future directions and challenges for the scientific and intelligence for comparison to the WM–intelligence relations. This research was partially study supported by Air Force Office of Scientific of the convergence of WMC, attention control, and intelligence. We then discuss the implications of the meta-analytic results in the Research Grant F49620-01-0170 (Phillip L. Ackerman, principal investicontext of both enduring psychometric measurement and theory gator). We thank the following individuals for providing correlations from issues. their empirical research: Scott Chaiken, Sandra Hale, Tim Salthouse, Lee Swanson,Theorists Bill Tirre,have and recently Werner Wittmann. We also thank Joni Lakin and WMC is primarily a domain-general construct; (c) WMC is more speculated that individual differences in Sarah working Molouki for their assistance obtaining correlations and data entry memory capacity in (WMC) may explain reasoning ability, closely related to Gf and reasoning than is short-term memory 1 and processing. It is interesting to note that in fact the original Stauffer, Ree, and general intelligence (Spearman’s g), or both (e.g., Engle, 2002; (STM). The latter two arguments are now well supported by Working Memory Capacity and Fluid Intelligence Are Strongly Related Constructs: Comment on Ackerman, Beier, and Boyle (2005) Correspondence concerning this article should be addressed to Phillip L. Caretta (1996) article does not state this conclusion. The higher order factor Figure 2. A nonhierarchical representation of g and working memory (WM) factors, based on a stru equation model of meta-analytically derived correlations among ability and WM measures. Kane, M.J., Conway, A.R.A., Hambrick, D.Z., & Engle, R.W. (2007). Variation in working memory capacity as variation in executive attention and control. In A.R.A. Conway, C.Jarrold, M.J. Kane, A. Miyake, and J.N. Towse (Eds.), Variation in Working Memory (pp. 21 - 48). NY: Oxford University Press. Cattell-Horn-Carroll Theory of Cognitive Abilities Controlled Motor Perception Attention Knowledge Fluid Gp Psychomotor Gk Gv Reasoning Gf Ga Gsm Gh Go Short-Term Abilities Gq Grw Gc Gkn Memory Gps Gt Gs Glr Psychomotor Speed Attentional Learning Efficiency Speed of Perception Fluency & Retrieval Fluency Speed (Shea, Lubinski, & Benbow, 2001; Wai, Lubinski, & Benbow, 2009; & Steel, 1979). A number of longitudinal studies based on Project Webb, Lubinski, & Benbow, 2007). Like Smith (1964) comprehenTALENT’s 11-year follow-up underscore the importance of spatial sive review ofSpatial spatial ability, Super and Bachrach (1957) document accomplishmentsand in STEM disciplines (Austin & Hanisch, ability and STEM: A sleeping giant forability talentforidentification development that exceptional general intellectual potential is characteristic of 1990; Gohm et al., 1998; Humphreys & Lubinski, 1996; Humphreys * Lubinski engineers andDavid physical scientists at early ages; however, they went & Yao, 2002; Humphreys et al., 1993). A recent study comparing onto stress that specific abilities – especially mathematical reasonthese data to TN modern longitudinal findings from the Study of Department of Psychology and Human Development, Peabody 0552, 230 Appleton Place, Vanderbilt University, Nashville, 37203, United States. ing and spatial ability – are also salient features of their individuMathematically Precocious Youth (SMPY; Lubinski & Benbow, ality. Moreover, they also noted the importance of scientific 2006), is especially relevant to understanding the development of a r t i c l e i n f o a b s t r a c t interests, and called for additional longitudinal research, for inSTEM talent (Wai et al., 2009). Articleyoung history: adolescents over 10- to Spatial ability for is a powerful sourceet of individual differences that has neglected in comstance, following 15-years ascer- systematicWai al. (2009) focused onbeen participants’ highest degree reReceived 27 January 2010 plex learning and work settings; it has also been neglected in modeling the development of expertise and taining how these and other personal attributes, and contrasting ceived (bachelor’s, master’s, or doctorate), the disciplines within Received in revised form 10 March 2010 creative accomplishments. Nevertheless, over 50 years of longitudinal research documents the important Accepted 18 March 2010 opportunities and supports, factor into differential development. which their degrees settings were wherein earned, and their occupations as a funcrole that spatial ability plays in educational and occupational sophisticated reasoning Available online 14 April 2010 with figures, and patterns, and shapestion is essential. Given the(‘‘g”) contemporary push for developing STEM (sci- spatial, and verDuring the years between Super and Bachrach (1957) Smith of general and specific (mathematical, ence, technology, engineering, and mathematics) talent in the information age, an opportunity is avail(1964), John Keywords: C. Flanagan et al. (1962) launched bal) abilities. Fig. ability. 1 graphs andresearch specific ability profiles able toProject highlight Talent, the psychological significance of spatial Doing the so is general likely to inform Spatial ability aptitude-by-treatment and Underwood’s idea to utilize individualterminal differences as which was expressly the kind of longitudinal on study that Super’sinteractions of Project Talent(1975) participants earning degrees in various STEM a crucible for theory construction. Incorporating spatial ability in talent identification procedures for Gifted NSF team envisioned. Because of its comprehensiveness and size, disciplines. Because highly congruent findings were observed for advanced learning opportunities uncovers an under-utilized pool of talent for meeting the complex needs Talent development longitudinal findings from Project TALENT areof an among the technological most all four cohorts, grades 9–12, the cohorts were combined. High ever-growing world; furthermore, selecting students for advanced learning opportuTalent searches nities in STEM without considering spatial ability might be iatrogenic. compelling for illustrating the role that spatial ability plays in general intelligence and an intellectual orientation dominated by ! 2010 Elsevier Ltd. All rights reserved. developing expertise in STEM. Project TALENT’s initial data collechigh mathematical and spatial abilities, relative to verbal ability, In celebrating the distinguished career of Thomas J. Bouchard, Jr., it seems fitting to recall a vivid memory of what it was like sitting in on Bouchard’s course in individual differences (or ‘‘IDs”), first as a student and, subsequently, as his graduate teaching assistant for the same course. Twice a week, students would listen to Bouchard analyze empirical studies and break them apart boneby-bone. Many would appear numb as their deeply held suppositions about human behavior were found to have little empirical basis. Like Donald G. Paterson’s course in IDs, which prior generations of Minnesota graduate students experienced, students left Bouchard’s course with new perspectives. Students having the privilege of experiencing one of these two intellectual giants – over the 80-year period they successively taught IDs at Minnesota – could never again look at human behavior through the same lens. Their newly acquired knowledge base forever changed how they viewed the human condition and the complex tensions surrounding social order, liberty, and individual differences (Wells, 1937). One of the more memorable things about Bouchard’s course was that, occasionally, after presenting a compelling empirical demonstration to the class, on the power that psychological variables can hold for predicting important socially-valued outcomes (educational achievements, occupational accomplishments, or life in general), Bouchard would turn to the class and say: ‘‘See, see, see what happens when psychologists choose to study real variables.” This point of view was not unrelated to that of Bouchard’s colleague Paul E. Meehl; Meehl would occasionally wonder out * Tel.: +1 615 343 1195. E-mail address: [email protected] loud whether it would be scientifically prophylactic to require graduate students in psychology to take a minor in a natural science like biology or genetics. By going so, Meehl speculated, they might be able to recognize a meaningful scientific contribution, if they should ever happen to encounter one in psychology! These two anecdotes set the stage for this contribution, regarding one of Bouchard’s favorite psychological variables, spatial ability. For decades, spatial ability has surfaced as a salient characteristic of young adolescents who go onto develop expertise in science, technology, engineering, and mathematics (STEM), yet applied psychologists and talent development researchers have failed to fully recognize this important dimension of human individuality.1 Contemporary discourse on the importance of identifying and nurturing STEM talent affords an opportunity to correct for this neglected ‘‘real variable.” 1. Spatial ability and STEM: decades of longitudinal research For example, during the same year Sputnik was launched, a little known report, Scientific Careers, was published on the psychological characteristics of individuals harboring STEM talent. This report was based on a NSF committee chaired by Donald Super, who assembled a distinguished team of psychologists (Harold 1 The referent generality of spatial ability extends well beyond STEM domains and encompasses among other things the creative arts in particular (Humphreys, Lubinski, & Yao, 1993). This article, however, will be restricted to STEM domains. For further reading on the educational and psychological significance of spatial ability, see Lohman (1988, 1994, 1996). 0191-8869/$ - see front matter ! 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.paid.2010.03.022 Fig. 1. Average z-scores of participants on spatial, math, and verbal ability for bachelor’s degrees, master’s degrees, and PhDs are plotted by field. The groups are plotted in rank order of their normative standing on g (S + M + V) along the x-axis and the line with the arrows from each field pointing to it indicates on the continuous scale where they are in general mental ability. This figure is standardized in relation to all participants with complete ability data at the time of initial testing. Respective N’s for each group (Males + Females) were (for bachelor’s, master’s, and doctorates respectively): engineering (1143, 339, and 71), physical science (633, 182, and 202), math/computer science (877, 266, and 57), biological science (740, 182, and 79), humanities (3226, 695, and 82), social science (2609, 484, and 158), arts (615, M = 171), business (2386, M + D = 191), and education (3403, M + D = 1505). *For education and business, masters and doctorates were combined because the doctorate samples for these groups were too small to obtain stability (N < 30). From Wai et al. (2009). Síntese das abordagens educacional (conhecimento) e psicológica (processos mentais) Conclusões … Conclusões • • Confusões atuais ao se tratar das competências cognitivas: • Tratadas com um único rótulo na literatura "QI" • Habilidades cognitivas incluídas nos sistemas em larga escala: Gc, Grw e Gq Comp do sec XXI enfatizam outros fatores amplos (no âmbito cognitivo): • Pensamento crítico e solução de problemas: Gf, Gv • Criatividade inovação: Glr Gf • Variáveis socioemocionais: O, C, A, N- • O modelo CHC é importante para mapear quais competências cognitivas estamos avaliando nos sistemas em larga escala • Será que deveríamos monitorar outras capacidades além daquelas já presentes nos sistemas em larga escala ? Qual seria a utilidade de tal sistema ? • Temas para Parte 2: Maleabilidade de g e WM, integração entre modelo CHC e a neuropsicologia Obrigado! [email protected]