3 edition of The effect of missing data treatment on Mantel-Haenszel DIF detection found in the catalog.
The effect of missing data treatment on Mantel-Haenszel DIF detection
Barnabas Chukwujiebere Emenogu
Published
2006
.
Written in
Test items that are differentially difficult for groups of examinees that are matched on the ability pose a problem for educational and psychological measurements. Such items are typically detected using differential item functioning (DIF) analyses, the most common of which is the Mantel-Haenszel method. Most implementations of the Mantel-Haenszel delete records from which any responses are missing or replace missing responses with scores of 0. This study examined the effect of these and other treatments for missing data in Mantel-Haenszel DIF analyses using data from the 1995 Trends in International Mathematics and Science Study (TIMSS) and the School Achievement Indicators Program (SAIP) 2001 Mathematics Assessment. Mantel-Haenszel DIF analyses were performed using a total score and a proportion score as matching variables and treating missing data by listwise deletion, analysiswise deletion, and scoring missing data as incorrect.Results of the analyses suggest that in the TIMSS dataset, where there were 41 dichotomously scored items and little missing data, matching based on the proportion score resulted in detecting more items showing significant values of DIF. However, in 80% of items all MDTs resulted in the same decision as to whether or not an item showed DIF. All missing data treatments identified the same magnitude and direction for 33% of the DIF items. In contrast, in the SAIP dataset, which had 75 items and more missing data, matching based on the total score resulted in detecting more items as showing significant values of DIF in favour of the reference group while matching based on proportion score led to detecting more DIF items in favour of the focal group. Of the 24 DIF items, the listwise deletion conditions identified only two while the other four conditions identified 22 with nine of them across all four conditions. However, all MDTs led to similar decisions in 68% of items. The results of this study clearly demonstrate the importance of decisions about how to treat missing data in DIF analyses.
Edition Notes
| Statement | by Barnabas Chukwujiebere Emenogu. |
| The Physical Object | |
|---|---|
| Pagination | x, 112 leaves : |
| Number of Pages | 112 |
| ID Numbers | |
| Open Library | OL21356531M |
| ISBN 10 | 9780494218112 |
| OCLC/WorldCa | 373314147 |
So, what is the effect of those 4 DIF items on the mean ability estimate of the focus group? The size of the overall DIF effect is (sum of 4 DIF effects)/ On a test of 15 items, this is almost certainly much less than the S.E. of a person measure, so it would be undetectable for an individual. But it could be noticeable for a whole group. One way to address this variation across studies is to perform a random-effects meta-analysis. In a random-effects meta-analysis we usually assume that the true effects are normally distributed. For example, in Figure the mean of all true effect sizes isbut theindividualeffect sizes are distributed aboutthis mean,as.
The classical Mantel-Haenszel method however only works for confounders with a limited number of discrete strata, which limits its utility, and appears to have no basis in statistical models. Here we revisit the Mantel-Haenszel method and propose an Cited by: 3. A common approach is based on scatter plots of the treatment effect estimated by individual studies versus a measure of study size or precision (the "funnel plot"). In this graphical representation, larger and more precise studies are plotted at the top, near the combined effect size, while smaller and less precise studies will show a wider Cited by:
Dichotomous data Continuous data Time-to-event data Types of data and effect measures are discussed in Chapter 9 (Section ). Unit of analysis issues Special issues in the analysis of studies with non-standard designs, such as cross-over trials and cluster-randomized trials, should be . The objective of this study was to examine the cross-cultural differences of the PANSS across six geo-cultural regions. The specific aims are (1) to examine measurement properties of the PANSS; and (2) to examine how each of the 30 items function across geo-cultural regions. Data was obtained for 1, raters from 6 different regions: Eastern Asia (n = Cited by: 6.
Planning evaluation
AICPA injunction case
Marketing
Introductory chemistry
Zagat New Orleans Restaurant Survey: 1990 Update (Zagat Survey: New Orleans Restaurants)
Malaysia and Brunei
Report on the Cumberland County Juvenile Assessment Center
Race relations in the South
Weldons practical knitter.
Basic Sightsinging and Ear Training
I am not a copycat!
Cakes, cookies and cake icings for quantity cookery
Selected essays and other short pieces of Jonathan Swift
Technological growth and social change
Most implementations of the Mantel-Haenszel differential item functioning procedure delete records with missing responses or replace missing responses with scores of 0. These treatments of missing data make strong assumptions about the causes of the missing data.
Such assumptions may be particularly problematic when groups differ in their patterns of by: 7. The performance of two simple and frequently used DIF methods that do not account for multilevel data structure, the Mantel-Haenszel test (MH) and logistic regression (LR), was compared with the.
In this study, it is aimed to investigate the impact of different missing data handling methods on the detection of Differential Item Functioning methods (Mantel Haenszel and Standardization.
Emenogu, B. C., Falenchuck, O., & Childs, R. The effect of missing data treatment on Mantel-Haenszel DIF detection. The Alberta Journal of Educational Author: Kuan-Yu Jin, Yi-Jhen Wu, Hui-Fang Chen.
irregular data points. This study applied smoothing techniques to frequency distributions and investigated the impact of smoothed data on the Mantel-Haenszel (MH) DIF detection in small samples.
Eight sample-size combinations were randomly drawn from a real data set to make the study realistic and were replicated 80 times to produce stable results. Mantel-Haenszel is the industry-standard DIF statistic, but it expects complete data because it stratifies the data by raw scores.
Please Google "Mantel-Haenszel". The Winsteps implementation is slightly different because it stratifies by person measure (same as raw scores for complete data), so it is robust against missing data. At the Educational Testing Service, the Mantel-Haenszel procedure is used for differential item functioning (DIF) detection, and the standardization procedure is used to describe DIF.
This report describes these procedures. First, an important distinction is made between DIF and Impact, pointing out the need to compare the comparable. The Effect of Missing Data Treatment on Mantel-Haenszel DIF Detection.
Most implementations of the Mantel-Haenszel differential item functioning procedure delete records with missing responses or replace missing responses with scores of 0. This article describes the results of a simulation study to investigate the impact of missing data.
The Mantel -Haenszel procedure is considered by some to be the most p owerful test for uniform DIF for dichotomous items (Holland & Thayer, ) The Mantel Haenszel procedure is easy to conduct, has an effect size measure and test of significance, and works well for small sample sizes However, the Mantel -Haenszel procedure detects uniform DIF.
Leadership, Higher and Adult Education Centre for the Study of Canadian and International Higher Education Research Overview Ruth Childs conducts research on the design and equity of large-scale assessments, admissions processes, and other evaluation systems.
effect of sample size, ability distribution and test length on detection of differential item functioning (DIF) using Mantel-Haenszel statistic. Objectives of the Study The objectives of the study were to: (i) Determine the effect of Sample Size, Ability Distribution and Test Length on the Effect Size of DIF items across 3 DIF Types; A, B and C.
The Mantel-Haenszel method is an approach for fitting meta-analytic fixed-effects models when dealing with studies providing data in the form of 2x2 tables or in the form of event counts (i.e., person-time data) for two groups (Mantel & Haenszel, ). of DIF or variances in ability distribution) and is an aspect that methodologists should consider in future simulation studies.
Keywords: Type I error, statistical power, Mantel-Haenszel, differential item functioning, meta-analysis Standardized measurement instruments or tests have become an. A new approach for differential item functioning detection using Mantel-Haenszel methods.
The GMHDIF program. Autores: Ángel Manuel Fidalgo Aliste Localización: The Spanish Journal of Psychology, ISSNVol. 14, Nº. 2,págs. Idioma: español DOI: /rev_sjopvn; Referencias bibliográficas. The effect of missing data treatment on Mantel-Haenszel DIF detection. The Alberta Journal of Educational Research, 56(4), Falenchuk, O., & Herbert, M.
().Author: Hüseyin Selvi, Devrim Özdemir Alıcı. The Mantel-Haenszel method • A popular DIF method since the late s; still stands as very effective compared with newer methods • Used by Educational Testing Service (ETS) in screening for uniform DIF • The MH method treats the DIF detection problem as one involving three-way contingency tables.
The threeFile Size: 1MB. For slightly less rare events, the Mantel-Haenszel fixed-effects odds-ratio (however, note: zero-cell corrections are a difficult topic, because you do not want to ignore trials with some zero cell, but a correction biases the result too much towards no effect) and logistic regression (standard, exact or Firth penalized likelihood logistic.
Robitzsch and Rupp () studied the effect of missing values on determining differential item functioning (comparison of Mantel–Haenszel and logistic regression techniques). However, in those studies, the examination of missing values or missing value procedure was different from those in the present study.
As described below, missingFile Size: KB. CDMSIB: Differential Item Functioning Detection Procedure in Cognitive Diagnostic Assessment. Paper presented at the annual meeting of the American Educational Research Association, Philadelphia.
Hou, L, Nandakumar, R, & de la Torre, J (, April). DIF in CDM: Comparing Wald Test with Mantel-Haenszel and SIBTEST.
four responses where the third response is missing. Note that DIFAS will not recognize a space as a missing value because a space is used as a delimiter in space-delimited files. DATA BEGINS ON FIRST LINE OF FILE. When reading in data, DIFAS assumes that the data begins in the first line of the file being read in.
Thus, a first lineFile Size: 73KB. R> plot(y ~ Latitude, data = BCG, ylab = "Estimated log-OR") R> abline(lm(y ~ Latitude, data = BCG, weights = studyweights)) 20 30 40 50 − − − Latitude Estimated log−OR Figure Plot of observed effect size for the BCGvaccine data against latitude, with a weighted least squares regression fit shown in Size: 74KB.Cochran-Mantel-Haenszel techniques: Applications involving epidemiologic survey data based on the difference between the observed and expected frequencies in each 2 x 2 table.
As with the classic chi-square test of independence in a single 2 x 2 table, it suffices to compare the observed and expected count in one cell per by: 7. Turnbull () concluded his early ETS treatment of fairness with the following statement: “Fairness, like its amoral brother, validity, resides not in tests or test scores but in the relation to its uses” (p.
4–5).While several ETS authors had addressed the relative lower performance of minority groups on tests of cognitive ability and its relationship to grades (e.g., Cited by: 9.
