Cognition and Development Assessment in Low Resource Conditions

Cognition and Development
Assessment in Low Resource
Conditions: Where are we?
What is new?
Elizabeth Prado
Program in International and Community Nutrition,
University of California at Davis, CA, USA
Summit Institute of Development, Mataram, Indonesia
Conceptual Model
Engle, P. L., Black, M. M., Behrman, J. R., de Mello, M. C., Gertler, P. J., Kapiriri, L., et al. (2007). Strategies to
avoid the loss of developmental potential in more than 200 million children in the developing world.
Lancet, 369, 229-242.
Common Challenges in LAMI Countries
1. Few or no tests exist that have been developed,
standardized, and normed in the local context
2. Data collectors have relatively low levels of education
and no previous experience assessing child
development
3. Low literacy levels in the population
4. Limited time and resources for pilot testing
5. Constraints on time for assessment (participant burden)
6. Constraints on location for assessment
•
•
may be crowded and noisy
may be impossible to use computers or special equipment
Framework
Underlying
Construct/Ability
Method
Implementation
Data:
Item Scores
Subscale Scores
Composite Scores
Risk/Delay
Conclusions depend
on how well the
scores reflect the
underlying
construct/ability
Underlying Construct/Ability
• What do we want to measure?
• What do we expect to be affected by the intervention?
• Previous behavioral studies
• Brain mechanisms
• What abilities are important for
• School performance
• Job performance
• Daily life
Carroll Model
Pase, M. P., & Stough, C. (2013). Describing a taxonomy of cognitive processes
for clinical trials assessing cognition. The American journal of clinical nutrition,
98(2), 509-510.
Cognitive Neuroscience Model
Hackman, D. A., & Farah, M. J. (2009). Socioeconomic status and the developing
brain. Trends in Cognitive Sciences, 13(2), 65-73.
Copying a shape with blocks
Test Name
Sub-Test Name
Domain Name
KABC-2
Triangles
Simultaneous
Processing/Visual Processing
(Gv)
WISC-IV
Block Design
Perceptual Reasoning
WPPSI-IV
Block Design
Visual Spatial
NEPSY-II
Block Construction Visuospatial Processing
Pattern
Construction
Spatial Ability (Gv)
BAS3
McCarthy
Scales
Block Building
Perceptual-performance
Test Validation
Underlying
construct/ability
What do we want to
measure?
e.g. The motor skills a
normally developing
child acquires at each
month of age
Validation
How do we know
that the method
measures the
construct?
e.g. Data from a large
normative sample on
what motor skills
children acquire at
each month of age
How do we measure
it?
e.g. Bayley scale of
motor development
Method
Standard score
55
70
85
100
115
130
145
Valid across contexts?
Bayley motor items in 6-month old German and
Cameroonian infants
Vierhaus, M., Lohaus, A., Kolling, T., Teubert, M., Keller, H., Fassbender, I., et al. (2011). The development
of 3- to 9-month-old infants in two cultural contexts: Bayley longitudinal results for Cameroonian and
German infants. European Journal of Developmental Psychology, 8(3), 349-366.
Types of Bias
• Construct Bias: the underlying construct is not identical across
cultural groups
• Method Bias: the familiarity of children with the test materials
and procedures is not identical across groups
• Item Bias: certain items function differently in different
cultural groups
Greenfield, P. M. (1997). You can't take it with you: Why ability assessments
don't cross cultures. American Psychologist, 52, 1115-1124.
Van de Vijver, F. & Tanzer, N. K. (2004). Bias and equivalence in cross-cultural
assessment: an overview. European Journal of Applied Psychology, 54, 119135.
Pilot Testing
• Involve local professionals (e.g. psychologists, teachers,
community workers) to advise on the appropriateness of items,
materials, questions
• Translation/Back-Translation
• Pilot Testing
• Make sure the participants understand the instructions/questions
• Check the distribution of scores (ceiling/floor effects)
Holding, P., Abubakar, A., & Kitsao-Wekulo, P. (2010). Where there are no tests:
A systematic approach to test adaptation. In M. L. Landow (Ed.), Cognitive
Impairment: Causes, Diagnosis, and Treatments: Nova Science Publishers, Inc.
Prado, E. L., Hartini, S., Rahmawati, A., Ismayani, E., Hidayati, A., Hikmah, N., et
al. (2010). Test selection, adaptation, and evaluation: a systematic approach to
assess nutritional influences on child development in developing countries. The
British journal of educational psychology, 80(Pt 1), 31-53.
Types of Assessments
Physiological
Measures
Brain
Structure
Brain
Function
Behavioral
Measures
Direct
assessment of
the child
Screening
Test
Ability
Test
Parent/Teacher
report
Screening
Test
Observation
(e.g. timesampling)
Ability
Test
Brain imaging
• Brain Structure
• Structural 3D Magnetic Resonance Imaging (MRI)
• Diffusion Tensor Imaging (DTI)
• Brain Function
• Functional Magnetic Resonance Imaging (fMRI)
• Magnetic Resonance Spectroscopy (MRS)
• Proton spectroscopy
• Phosphorus spectroscopy
• Electroencephalography (EEG) / Event-Related Potentials (ERP)
• Functional Near Infrared Spectroscopy (fNIRS)
Sizonenko, S. V., Babiloni, C., de Bruin, E. A., Isaacs, E. B., Jonsson, L. S., Kennedy,
D. O., et al. (2013). Brain imaging and human nutrition: which measures to use in
intervention studies? The British journal of nutrition, 110 Suppl 1, S1-S30.
fNIRS and EEG/ERP
• fNIRS
• Measures blood flow in the
cortex in response to an
auditory or visual stimulus
• ERP
• Measures electrical
currents in response to an
auditory or visual stimulus
• Good temporal resolution,
but poor spatial resolution
compared to fMRI
Lloyd-Fox, S., Papademetriou, M., Darboe, M. K., Everdell, N. L., Wegmuller, R.,
Prentice, A. M., et al. (2014). Functional near infrared spectroscopy (fNIRS) to
assess cognitive function in infants in rural Africa. Scientific reports, 4, 4740.
Commonly Used Behavioral Tests:
0-3+ years
Screening Test
Direct Assessment
Parent/Teacher Report
• Denver Developmental
Screening Test
• Ages & Stages
Questionnaire (ASQ)
• Parents’ Evaluation of
Development Status (PEDS)
• MacArthur Communicative
Development Inventory
(CDI)
• Infant and Toddler SocioEmotional Assessment
(ITSEA)
Fernald, L. C. H., Kariger, P., Engle, P., & Raikes, A. (2009). Examining early child
development in low-income countries: A toolkit for the assessment of children in
the first five years of life. Washington, DC: The World Bank.
Frongillo, E. A., Tofail, F., Hamadani, J. D., Warren, A. M., & Mehrin, S. F. (2014).
Measures and indicators for assessing impact of interventions integrating
nutrition, health, and early childhood development. Annals of the New York
Academy of Sciences, 1308, 68-88.
Ability Test
• Bayley Scales of Infant
Development
• Griffiths Mental
Development Scales
Commonly Used Behavioral Tests:
3-18 years
Ability Test
Direct Assessment
Parent/Teacher Report
• British Ability Scales (BAS)
/Differential Ability Scales
(US version)
• Kauffman Assessment
Battery for Children (KABC)
• Wechsler Intelligence Scale
for Children (WISC)
• McCarthy Scales
• Peabody Picture Vocabulary
Test (PPVT)
• Child Behavior Checklist
(CBCL)
• Strengths and
Difficulties
Questionnaire (SDQ)
• Behavior Rating
Inventory of Executive
Function (BRIEF)
Fernald, L. C. H., Kariger, P., Engle, P., & Raikes, A. (2009). Examining early child
development in low-income countries: A toolkit for the assessment of children in
the first five years of life. Washington, DC: The World Bank.
Frongillo, E. A., Tofail, F., Hamadani, J. D., Warren, A. M., & Mehrin, S. F. (2014).
Measures and indicators for assessing impact of interventions integrating
nutrition, health, and early childhood development. Annals of the New York
Academy of Sciences, 1308, 68-88.
Examples of Tests Developed in
LAMI Countries
Direct Assessment
Parent/Teacher Report
Screening Test
• Rapid
Neurodevelopmental
Assessment (RNDA):
Bangladesh
• Ten Questions
Questionnaire (TQQ)
• Guide for Monitoring Child
Development: Turkey
Ability Test
• Grover-Counter Scale of • Profile of Socio-Emotional
Cognitive Development:
Development (PSED): Kenya
South Africa
• Developmental Milestones
• Malawi Developmental
Checklist (DMC): Kenya
Assessment Tool
(MDAT)
• Kilifi Developmental
Inventory (KDI): Kenya
Fernald, L. C. H., Kariger, P., Engle, P., & Raikes, A. (2009). Examining early child
development in low-income countries: A toolkit for the assessment of children in
the first five years of life. Washington, DC: The World Bank.
New Initiatives for Open-Source Tools
• NIH Toolbox for Neurological Assessment
• Age 3-85 years
• 4 Domains: Cognition, Emotion, Motor, Sensation
• Population-Level Measures
• Early Development Instrument (age 4-7 years)
• UNICEF Multiple Indicator Cluster Survey Early Child Development
Index (age 3-4 years)
• Work in Progress
• Global Child Development Group Scalable Early
Childhood Indicators
• UNESCO Holistic Early Child Development Index
• Learning Metrics Task Force (age 4-6 years)
• WHO indicators for tracking ECD (age 0-3 years)
Psychosocial and Biological Risk Factors
Engle, P. L., Black, M. M., Behrman, J. R., de Mello, M. C., Gertler, P. J., Kapiriri, L., et al. (2007). Strategies to
avoid the loss of developmental potential in more than 200 million children in the developing world.
Lancet, 369, 229-242.
Psychosocial and Biological Risk Factors
• Developmental support from the environment
• Home Observation for the Measurement of the Environment
(HOME) Inventory
• Family Care Indicators
• Maternal mental health
• Self-Report Questionnaire
• Centre for Epidemiological Studies Depression Test
• Dietary Intake
• Anthropometry
• Anemia
Frongillo, E. A., Tofail, F., Hamadani, J. D., Warren, A. M., & Mehrin, S. F. (2014).
Measures and indicators for assessing impact of interventions integrating
nutrition, health, and early childhood development. Annals of the New York
Academy of Sciences, 1308, 68-88.
Framework
Underlying
Construct/Ability
Method
Implementation
Data:
Item Scores
Subscale Scores
Composite Scores
Risk/Delay
Conclusions depend
on how well the
scores reflect the
underlying
construct/ability
Kilifi Developmental Inventory
Manual
Let's jump.
• Demonstrate jumping with both feet leaving
the ground.
Jump.
Jump again.
Now jump alone.
• Practice with the child.
• Providing necessary support such as holding
the child's hand and jumping with them,*or
letting them hold onto an object
• Encourage the child to do it without support.
• Encourage the child to jump.
• Observe any jumps in a row the child can
make and score for LM07 and 08.
Implementation
• Manuals and Training Materials
• Tester/Interviewer Evaluation
• Knowledge and practice-based assessments
• Inter-Rater agreement
• Test-Retest reliability
• Ongoing Evaluation and Quality Control
• Continued periodic assessments
• Data review and feedback
Shankar, A. V., Asrilla, Z., Kadha, J. K., Sebayang, S., Apriatni, M., Sulastri, A., et al.
(2009). Programmatic effects of a large-scale multiple-micronutrient
supplementation trial in Indonesia: using community facilitators as intermediaries
for behavior change. Food and Nutrition Bulletin, 30(2 Suppl), S207-214.
Conculsion
• Measuring cognition and child development in LAMI countries
is challenging but feasible and important
• Key elements:
•
•
•
•
•
Collaboration with psychologists/neuroscietnists
Input from local professionals
Pilot testing and adaptation
High quality manuals and training and evaluation of testers
Evaluation and reporting of data quality
•
•
•
•
•
Distribution of scores
Internal reliability
Inter-rater agreement
Test-retest reliability
Correlation with expected variables
Thank you!