The Use of Measurement Invariance with Dichotomous Variables as Evidence of Validity
DOI:
https://doi.org/10.35670/1667-4545.v18.n2.20807Keywords:
validity, fairness, measurement invariance, dichotomous items, Academic Aptitude TestAbstract
This paper explains and exemplifies how an analysis of the measurement invariance with dichotomous items may be implemented in order to obtain evidence of validity. The example was developed with data from the Academic Aptitude Test (PAA, for its initials in Spanish; PPPAA, 2014) at the University of Costa Rica (UCR). Measurement invariance was analyzed according to two classifications, by high school kind (public or private) and by sex. The method used was a confirmatory factor analysis with two correlated factors defined from the two components of the PAA (N = 11304). Each group analysis presented strict invariance. It was concluded that the test scores have the same unit of measurement for both men and women, and among students from public and private high schools. Finally, these results constitute evidence of validity of the PAA scores in terms of their internal structure from the perspective of fairness.
Downloads
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238-246. doi: 10.1037//0033-2909.107.2.238
Bollen, K. A. (1989). Structural Equations with Latent Variables. New York: Wiley. doi: 10.1002/9781118619179
Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford.
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464-504. doi: 10.1080/10705510701301834
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233-255. doi: 10.1207/s15328007sem0902_5
Cumsille, P., Martínez, M. L., Rodríguez, V., & Darling, N. (2014). Análisis psicométrico de la Escala Parental Breve (EPB): Invarianza demográfica y longitudinal en adolescentes chilenos. Psykhe, 23(2), 1-14. doi: 10.7764/psykhe.23.2.665
Desa, D. (2014). Evaluating measurement invariance of TALIS 2013 Complex Scales: Comparison between continuous and categorical multiple-group confirmatory factor analyses. OECD Education Working Papers, 103, 2-39. Paris: OECD Publishing. doi: 10.1787/5jz2kbbvlb7k-en
Guedea, J. C., Ornelas, M., Rodríguez, J. M., & Gastélum, G. (2012). Invarianza factorial de la Escala de Ansiedad Asociada a la Imagen Corporal en estudiantes universitarios de educación física y ciencias de la salud. Formación Universitaria, 5(6), 39-50. doi: 10.4067/s0718-50062012000600005
Hartwig, J., & Gregg, N. (2007). The role of extended time on the SAT reasoning test for students with disabilities and/or Attention-Deficit/Hyperactivity Disorder. Learning Disabilities Research & Practice, 22(2), 85-95. doi: 10.1111/j.1540-5826.2007.00233.x
Hortensius, L. (2012). Project for Introduction to Multivariate Statistics: Measurement Invariance. Retrieved from https://pdfs.semanticscholar.org/6d69/f889d6acd9a58706e8e230e4e43ab4ae3bda.pdf
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidsciplinary Journal, 6(1), 1-55. doi: 10.1080/10705519909540118
Jiménez, K., & Morales, E. (2010). Validez predictiva del promedio de admisión de la Universidad de Costa Rica y sus componentes. Actualidades en Psicología, 23(110), 21-55. doi: 10.15517/ap.v23i110.11
Kaplan, D. (2009). Structural Equation Modeling: Foundations and Extensions (2nd ed.). Thousand Oaks, CA: SAGE Publications. doi: 10.4135/9781452226576
Kosiol, N. A. (2010). Evaluating measurement invariance with censored ordinal data: A Monte Carlo comparison of alternative model estimators and scales of measurement (Master's thesis). Retrieved from https://digitalcommons.unl.edu
Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525-543. doi: 10.1007/bf02294825
Milfont, T. L., & Fischer, R. (2010). Testing measurement invariance across groups: Applications in crosscultural research. International Journal of Psychological Research, 3(1),111-121. doi: 10.21500/20112084.857
Millsap, R. E. (2007). Invariance in measurement and prediction revisited. Psychometrika, 72(4), 461-473. doi: 10.1007/s11336-007-9039-7
Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39(3), 479-515.
Montero-Rojas, E., Villalobos-Palma, J., & Valverde-Bermúdez, A. (2014). Factores institucionales, pedagógicos, psicosociales y sociodemográficos asociados al rendimiento académico en la Universidad de Costa Rica: Un análisis multinivel. Relieve, 13(2), 215-234. doi: 10.7203/relieve.13.2.4208
Muthén, L. K., & Muthén, B. O. (1998). Mplus user’s guide. Los Angeles, CA: Author.
Piqueras, J. A., Olivares, J., Vera-Villarroel, P., Marzo, J. C., & Kuhne, W. (2012). Invarianza factorial de la Escala para la Detección de Ansiedad Social (EDAS) en adolescentes españoles y chilenos. Anales de Psicología, 28(1), 203-214. Retrieved from http://www.redalyc.org/articulo.oa?id=16723161023
Pornprasertmanit, S. (2015). Package semTools. Retrieved in 2015 from http://cran.rproject.org/web/packages/semTools/semTools.pdf
PPPAA (2014). Prueba de Aptitud Académica. In Smith, V.(Ed.), Compendio de Instrumentos de Medición del IIP-2014.
R Core Team (2014). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing: Vienna, Austria.
Rock, D. A., Werts, C., & Grandy, J. E. (1981) Construct validity of the GRE Aptitude Test across populations-An empirical confirmatory study (Report 81-57). ETS. doi: 10.1002/j.2333-8504.1981.tb01284.x
Rojas-Torres, L. (2013). Validez predictiva de los componentes del promedio de admisión a la Universidad de Costa Rica utilizando el género y el tipo de sexo como variables control. Actualidades Investigativas en Educación, 13(1), 1-24. doi: 10.15517/aie.v13i1.11707
Rojas-Torres, L. (2014). Evidencias de validez de la Prueba de Aptitud Académica de la Universidad de Costa Rica basadas en su estructura interna. Actualidades en Psicología, 28(116), 15-26. doi: 10.15517/ap.v28i116.14889
Rosseel, Y. (2012). Lavaan: An R package for Structural Equation Modeling. Journal of Statistical Software, 48(2), 1-36. doi: 10.18637/jss.v048.i02
Sideridis, G. D., Tsaousis, I., & Al-harbi, K. A. (2015). Multipopulation invariance with dichotomous measures: Combining multi-group and MIMIC methodologies in evaluating the General Aptitude Test in the Arabic language. Journal of Psychoeducational Assessment, 33(6), 568-584. doi: 10.1177/0734282914567871
Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In K. J. Bryant, M. Windle & S. G. West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281-324). Washington, DC: American Psychological Association. doi: 10.1037/10222-009
Downloads
Published
Issue
Section
License
Copyright (c) 2018 Luis Rojas Torres, Guaner Rojas Rojas, Armel Brizuela Rodríguez
This work is licensed under a Creative Commons Attribution 4.0 International License.
Revista Evaluar aplica la Licencia Internacional de Atribuciones Comunes Creativas (Creative Commons Attribution License, CCAL). Bajo esta licencia, los autores retienen la propiedad de copyright de los artículos pero permiten que, sin que medie permiso de autor o editor, cualquier persona descargue y distribuya los artículos publicados en Evaluar. La única condición es que siempre y en todos los casos se cite a los autores y a la fuente original de publicación (i.e. Evaluar). El envío de artículos a Evaluar y la lectura de los mismos es totalmente gratuito.
