Analysis of test results and test quality during certification of non-destructive testing specialists

Cover Page

Cite item

Full Text

Open Access Open Access
Restricted Access Access granted
Restricted Access Subscription or Fee Access

Abstract

This article analyzes the results of personal certification for eddy current testing of railway facilities. The influences of education level, the work experience, the candidate age on the qualifying examinations results are considered. The conformity between test task difficulties and certification candidate training is analyzed with Rasch method. The characteristic curves of test task difficulties and the characteristic curves of candidate training levels are constructed. The results general and special examinations of 2020 and 2021 are analyzed to identify extreme test questions. The research results enable to assess the candidate training and the test task difficulties through latent parameters measured as logit unit. The analysis identified test tasks with overstated and understated task difficulty logits compared with the logits of the candidate training, assessed the correspondence between test difficulties and training candidate level, and assessed a distribution uniformity of the tasks’ difficulty. The required training logit of candidates for the certification is calculated for successful passing general and special exam tests. The getting results give a basis for further analysis of the tasks complexity, identification the reasons for the insufficiency of the auditory material familiarization for the tasks with low rate of response, for making adjustments to the consulting sessions, as well as the content and form of the test.

Full Text

Restricted Access

About the authors

V. V. Murav’ev

Kalashnikov Izhevsk State Technical University; Udmurt Federal Research Center Ural Branch Russian Academy of Sciences

Author for correspondence.
Email: vmuraviev@mail.ru
Russian Federation, 426069, Izhevsk, st. Studencheskaya, 7; 426067 Izhevsk, st. Tatyana Baramzina, 34

L. V. Volkova

Kalashnikov Izhevsk State Technical University

Email: vmuraviev@mail.ru
Russian Federation, 426069, Izhevsk, st. Studencheskaya, 7

O. V. Murav’eva

Kalashnikov Izhevsk State Technical University; Udmurt Federal Research Center Ural Branch Russian Academy of Sciences

Email: vmuraviev@mail.ru
Russian Federation, 426069, Izhevsk, st. Studencheskaya, 7; 426067 Izhevsk, st. Tatyana Baramzina, 34

S. A. Murashov

Kalashnikov Izhevsk State Technical University

Email: vmuraviev@mail.ru
Russian Federation, 426069, Izhevsk, st. Studencheskaya, 7

References

  1. Berrío Ángela I., Gomez-Benito J., Arias E. M. Developments and Trends in Research on Methods of Detecting Differential Item Functioning // Educational Research Review. 2020. V. 31. P. 100340. doi: 10.1016/j.edurev.2020.100340
  2. Boon H.J., Lewthwaite B. Development of an Instrument to Measure a Facet of Quality Teaching: Culturally Responsive Pedagogy // International Journal of Educational Research. 2015. V. 72. P. 38—58. doi: 10.1016/j.ijer.2015.05.002
  3. Aghekyan R. Validation of the SIEVEA Instrument Using the Rasch Analysis // International Journal of Educational Research. 2020. V. 103. P. 101619. doi: 10.1016/j.ijer.2020.101619
  4. Kruit P., Oostdam R., van den Berg E., Schuitema J. Performance Assessment as a Diagnostic Tool for Science Teachers // Research in Science Education. 2020. V. 50. No. 3. P. 1093—117. doi: 10.1007/s11165-018-9724-9
  5. Vaughan B. A Rasch Analysis of the Revised Study Process Questionnaire in an Australian Osteopathy Student Cohort // Studies in Educational Evaluation. 2018. V. 56. P. 144—53. doi: 10.1016/j.stueduc.2017.12.003
  6. Ehrich J.F., Woodcock S., West C. The Effect of Gender on Teaching Dispositions: A Rasch Measurement Approach // International Journal of Educational Research. 2020. V. 99. P. 101510. doi: 10.1016/j.ijer.2019.101510
  7. Zhao Yue, Huen J.M., Chan Y.W. Measuring Longitudinal Gains in Student Learning: A Comparison of Rasch Scoring and Summative Scoring Approaches // Research in Higher Education. 2017. V. 58. No. 6. P. 605—16. doi: 10.1007/s11162-016-9441-z
  8. Sabetova T.V. The use of latent variable method in human resource management // Russian Journal of Agricultural and Socio-Economic Sciences. 2017. V. 66. No. 6. P. 123—29. doi: 10.18551/rjoas.2017-06.14
  9. Van Zile-Tamsen C. Using Rasch Analysis to Inform Rating Scale Development // Research in Higher Education. 2017. V. 58. No. 8. P. 922—33. doi: 10.1007/s11162-017-9448-0
  10. Jääskä E., Aaltonen K. Teachers’ Experiences of Using Game-Based Learning Methods in Project Management Higher Education // Project Leadership and Society. 2022. V. 3. P. 100041. doi: 10.1016/j.plas.2022.100041
  11. Chang W.-Ch., Ludlow L., Grudnoff L., Ell F., Haigh M., Hill M. Measuring the Complexity of Teaching Practice for Equity: Development of a Scenario-Format Scale // Teaching and Teacher Education. 2019. V. 82. P. 69—85. doi: 10.1016/j.tate.2019.03.004
  12. Krasova N.A., Aleynikova N.E. Mathematical modeling of the problem of determining a set of control measures using the fuzzy method of analyzing hierarchies and a method based on measuring latent variables // Modeling, Optimization and Information Technologies. 2019. V. 7. No. 3 (26). P. 33. doi: 10.26102/2310-6018/2019.26.3.024
  13. Korypaeva Yu.V., Krasova N.E., Peshkov V.V., Ryabov S.V. Methodology for organizing and assessing the quality of research work of cadets of the military scientific society based on the Rush mathematical model // Bulletin of Voronezh State Technical University. 2020. V. 16. No. 2. P. 65—70. doi: 10.25987/VSTU.2020.16.2.010
  14. Murav’ev V.V., Volkova L.V., Platunov A.V., Bulatova E.G., Bajkalova T.N. Analysis of test results mean specialists in magnetic non-destructive testing for the certification // Kontrol. Diagnostika. 2015. No. 10. P. 36—42. EDN UJZFEL. doi: 10.14489/td.2015.010.pp.036-042.
  15. Muravjev V.V., Muravjeva O.V., Volkova L.V., Bogdan O.P., Platunov A.V., Bajkalova T.N. Test quality analysis during specialist certification on acoustic emission nondestructive testing of railway objects // In the world of non-destructive testing. 2015. V. 18. No. 3. P. 66—71. EDN UDJSQX.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download
2. Fig. 1. Results of assignment of certification levels (a) and level II certification for different types of nondestructive testing (b)

Download (384KB)
Indexing metadata
3. Fig. 2. Result of passing general and special examinations depending on length of service (a) and certification of specialists depending on length of service (b)

Download (449KB)
Indexing metadata
4. Fig. 3. Distribution of candidates for certification by education level

Download (168KB)
Indexing metadata
5. Fig. 4. Result of passing qualification exams depending on the level of education

Download (239KB)
Indexing metadata
6. Fig. 5. The result of passing qualification exams depending on the age of the subjects

Download (267KB)
Indexing metadata
7. Fig. 6. Test results for the eddy current general exam in 2021 (a) and 2022 (b)

Download (661KB)
Indexing metadata
8. Fig. 7. Distribution of correct answers for individual test questions of the general exam in 2021 (a) and 2022 (b)

Download (308KB)
Indexing metadata
9. Fig. 8. Results of the distribution of candidates' preparation level and difficulty of the general exam tasks in 2021 (a) and 2022 (b)

Download (413KB)
Indexing metadata
10. Fig. 9. Test results of candidates for the eddy current specialty certification exam in 2021 (a) and 2022 (b)

Download (745KB)
Indexing metadata
11. Fig. 10. Distribution of correct answers for individual test questions of the special eddy current control examination in 2021 (a) and 2022 (b)

Download (294KB)
Indexing metadata
12. Fig. 11. Distribution of the level of preparedness of candidates for certification and difficulty of tasks of the special eddy current inspection exam in 2021 (a) and 2022 (b)

Download (386KB)
Indexing metadata
13. Fig. 12. Characteristic test curves of the general exam (a) and certification candidate preparedness (b) in 2021-2022

Download (1016KB)
Indexing metadata
14. Fig. 13. Characteristic curves of test items of the general exam (a) and subjects (b)

Download (1MB)
Indexing metadata
15. Fig. 14. Characteristic curves of the special exam test (a) and subjects (b)

Download (1020KB)
Indexing metadata

Copyright (c) 2024 Russian Academy of Sciences