The Institute of National Education uses AI to "feel the pulse" of the national examination. Nearly half of the examination subjects have doubtful questions. It is recommended that the examination unit should establish a database and monitoring mechanism.
(Provided by the Examination Institute)
[Reporter Lin Xiaoyun/Taipei Report] The National Institute of Education of the Ministry of Education "checked the pulse" of the national examination, made good use of the power of AI technology, and completed the current situation and impact analysis of doubtful questions. It was found that nearly half (48.14%) of the examination questions had doubtful questions, of which 9 There are 3 question questions (internal) in the Chengkao section, which are constructed through the R platform and related packages, gradually imported and connected to the current system of the examination department, and assist the testing unit to establish a database and monitoring mechanism to protect the rights and interests of about 400,000 candidates every year.
Lin Chongxi, President of the National Education Institute, and Xu Shuxiang, the Minister of Examinations, provided professional assistance with the Test and Evaluation Research Center of the National Education Institute, aiming at the "correction rate of national examination test questions and its impact on the validity and reliability of the test - using medical expertise 10-month task-based research was carried out on "Personnel Examination as an Example", "Research on the Implementation Method of Policy Appropriateness for Comparison of Similar Questions in National Examination", and "Discussion on the Appropriateness of Subjects and Question Types of Civil Service Examination".
Please read on...
The National Academy of Education analyzed the results of doubtful questions. Among the 912 test subjects, there were 473 subjects with 0 doubtful questions, accounting for 51.86%. 48.14%, and more than 90% of the test subjects held, the number of doubtful questions is mostly within 3 questions (including).
In addition, in the past 10 years, except for dentists (1), dentists (2), and medical examiners, due to the relatively small number of doubtful questions over the years, most of them remained within the fixed range of doubtful questions, and there was no obvious downward trend. There is a downward trend in the number of doubtful questions in many subjects.
In addition, the study found that correcting scores for doubtful questions over the years mostly changed the test taker from "fail" to "pass", but did not change the test taker from "pass" to "fail". Equity impact is small.
As for the controversy over archaeological questions, most agree that "derivative questions and duplicate questions" can be directly used in the examinations of the selection section, but most think that "same questions or part of the same questions" should be avoided in order to maintain the quality of the test questions.
The National Academy of Education also suggested that the Examination Department should establish the principle of "selection of question types" according to the type of test. Most of the interviewed scholars and experts agree that the content of the multi-dimensional detailed table can be defined according to the "test type", which is divided into knowledge content and cognitive process. , and then consider practical factors, and choose the appropriate corresponding examination question type.
The most important thing is that the database and data-oriented monitoring mechanism have been established year by year. The National Education Institute has successively completed the "current situation and impact analysis of doubtful questions, classical test parameters, IRT parameter analysis and other procedures" for the examination department. Through the R platform and related packages Construction is carried out, and it is also gradually introduced into the current system of the Connection Examination Department, as a test unit to establish a database and monitoring mechanism.