I am struggling to identify two specific circumstances for which CTT and IRT might provide meaningful benefits to my test on relationship happiness and financial responsibility, and why. What can you recommend?
Here is my current post
Post by Day 4 an explanation of the advantages and disadvantages of CTT and IRT. Describe two specific circumstances for which IRT might provide meaningful benefits if applied to the development or implementation of the test that you are proposing for your Final Project and explain why. Finally, describe two specific circumstances for which CTT might provide meaningful benefits to your test, and explain why. Support your response using the Learning Resources and the current literature.
Mathematical models have been widely used in the analysis of educational and psychological test data (Hambleton & Swaminathan, 2013). Each model includes an equation for linking the respondent’s item performance with a latent trait and assumptions (Hambleton & Swaminathan, 2013). Item Response Theory (IRT) and Classical Test Theory CTT) are two of these models.
IRT (Item Response Theory)
Item Response Theory also known as Latent Response Theory or Latent Trait Theory is a mathematical model used to compare latent traits with their characteristics (Anastasi & Urbina, 1997). When looking at cognitive tests, latent traits are categorized as the abilities measured by the test (Anastasi & Urbina, 1997). When the total scores are collected, these scores are considered the first estimate of the ability or trait (Anastasi & Urbina, 1997). IRT models use different mathematical equations that are based on the diverse set of assumptions but generally, the results gathered with the different models are similar as long as the assumptions are met (Anastasi & Urbina, 1997). The goal of IRT is to demonstrate both invariant item statistics and trait approximations (Hambleton & Swaminathan, 2013). Plainly stated, IRT looks at the connection between the trait and the difficulty level of the test the person is taking. Thissen & Steinberg (1988) called IRT “mathematically sophisticated”. Some examples of the most common IRT models are latent linear, perfect scale, one, two, three-parameter normal Ogive, and continuous response (Hambleton & Swaminathan, 2013).
Advantages of Item Response Theory are questionnaire design, including strong methods for item choice and scale reduction (Embretson & Reise, 2013). Assuming we can measure a large pool of people on the same trait another advantage is the independence gained from sample test items (Hambleton & Swaminathan, 2013). This is technically described as invariance of item parameters providing a unified measurement in different groups (Anastasi & Urbina, 1997). As well as enabling instruments to be linked or cross-calibrated (Embretson & Reise, 2013). Being able to identify the descriptors like, item difficulty and discrimination indices is another advantage (Hambleton & Swaminathan, 2013). IRT also offers a “natural way of creating computer-adaptive tests” (Embretson & Reise, 2013). Which gives researchers the ability to make statistical adjustments to test scores (Van Der Linden & Hambleton, 2013).
Some disadvantages of IRT are rigorous assumptions, large sample size, considerations of differences between the delivery conditions of the test, and the final form may impact conditions that affect item statistics (American Educational Research Association, 2014). Examples of this might be test-taker motivation, time limits, test length, and mode of testing (American Educational Research Association, 2014).
For my final project IRT may be useful in narrowing down the traits I am trying to target, for instance, a partner’s frugality or level of commitment to the relationship.
CTT (Classical Test Theory)
Classical Test Theory (CTT) is known for the high quality of psychometrically sound scales that was founded by Charles Spearman (Kline, 2005). And has been the foundation for measurement theory for over 80 years (Kline, 2005). Most rules of measurement for psychologists are based on CTT (Embretson & Reise, 2013). Just like IRT, there are different types of CTT. Defined as a psychometric theory, that is based on an individual’s observed score and the total of the true score component and the independent random error component of the test taker (American Educational Research Association, 2014). Reliability of CTT is the correlation between scores on two comparable forms of a test assuming that taking one form has no impact on the other (American Educational Research Association, 2014).
One advantage of CTT is the ability to handle unequal categories for test development (Embretson & Reise, 2013). CTT is also the most common model for scale development and validation (Kline, 2005). Another advantage is that the longer the test the more reliability increases (Kline, 2005).
Some disadvantages of CTT are the prime concern of CTT to manage effectively with the random error portion (E) of the raw score (Kline, 2005). This theory has difficulty comparing scores across two different tests because they are not on the same scale and it assumes equal errors of measurement at all levels of ability (Kline, 2005).
Research has found that IRT is better than CTT in individual change detection, as long as the tests have at least 20 items (Emons, Jabrayilov, & Sijtsma, 2016). But for shorter tests, CTT is better than IRT at identifying a change in respondents (Emons, Jabrayilov, & Sijtsma, 2016).
Reference
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). Upper Saddle River, NJ: Prentice Hall.
Embretson, S., & Reise, S. (2013). Item response theory. Psychology Press.
Emons, W., Labrayilov, R., & Sijtsma, K. (2016). Comparison of Classical Test Theory and Item Response Theory in Individual Change Assessment. Applied Psychological Measurement 40(8): 559–572. doi: 10.1177/0146621616664046
Hambleton, R. K., & Swaminathan, H. (2013). Item response theory: Principles and applications. Springer Science & Business Media.
Kline, T. (2005). Psychological Testing: A Practical Approach to Design and Evaluation. SAGE Publications, Inc. https://doi-org.ezp.waldenulibrary.org/10.4135/9781483385693
Thissen, D., & Steinberg, L. (1988). Data analysis using item response theory. Psychological Bulletin, 104(3), 385–395. Retrieved from the Walden Library databases
Van Der Linden, W. J., & Hambleton, R. K. (2013). Handbook of modern item response theory. Springer Science & Business Media.
0 comments