Tools in Measuring Program and Learning Outcomes of Medical Students

JavaScript isn't enabled in your browser, so this file can't be opened. Enable and reload.

Tools in Measuring Program and Learning Outcomes of Medical Students – A Guide in Scoring and Grading (Part 3)

Subtitle: Tools in Measuring Knowledge, Skill and Attitude of Medical Students - A Guide in Scoring and Grading (Part 3)

Recommended Prior Study: Assessment and Evaluation of Medical Students – ROJoson TPORs; Tools in Measuring Program and Learning Outcomes of Medical Students – A Guide in Designing the Master Blueprint (Part 1); Tools in Measuring Program and Learning Outcomes of Medical Students – A Guide in Designing the Tools (Part 2)

Recommended Prior Study: Assessment and Evaluation of Medical Students – ROJoson TPORs

https://basicgeneralistphphysician.wordpress.com/2023/08/02/assessment-and-evaluation-of-medical-students-rojoson-tpors/;

https://basicgeneralistphphysician.wordpress.com/2023/08/11/tools-in-measuring-program-and-learning-outcomes-of-medical-students-a-guide-in-designing-a-master-blueprint-part-1-manuscript/

https://basicgeneralistphphysician.wordpress.com/2023/08/16/tools-in-measuring-program-and-learning-outcomes-of-medical-students-a-guide-in-designing-the-tools-part-2/

Note: Completed the plans of ROJ-OCIL on Medical Education on the topic: Tools in Measuring Program and Learning Outcomes of Medical Students (3 parts) - will amplify on the contents of each part in the future.

Tools in Measuring Program and Learning Outcomes of Medical Students – A Guide in Designing a Master Blueprint (Part 1)

Tools in Measuring Program and Learning Outcomes of Medical Students – A Guide in Designing the Tools (Part 2)

Tools in Measuring Program and Learning Outcomes of Medical Students – A Guide in Scoring and Grading (Part 3)

This is part of ROJ-OCIL-OLETE on Medical Education. ROJ = ROJoson; OCIL= Online Collaborative and Interactive Learning; OLETE = Online Learning cum Evaluation Test Exercise.

This ROJ-OCIL-OLETE is associated with an ROJoson Talk with the same title (see link: https://basicgeneralistphphysician.wordpress.com/2023/08/26/tools-in-measuring-program-and-learning-outcomes-of-medical-students-a-guide-in-scoring-and-grading-part-3/)

Basically, OLETE is an online test and/or exercise that has for its objectives the simultaneous promotion of learning, assessment and evaluation of the learners or students. Among the three objectives, promotion of learning is intended to be most important in my design for the OLETE. The assessment and evaluation is intended to be more of a self-assessment-evaluation of the students to reinforce their learning. Although the teacher-facilitator can use the OLETE to assess the pretest competency status (as will be reflected in the first attempt) and evaluate achievement in the subsequent attempts of the students, it is not the intention in the design of the OLETE to identify which students to flank or which to recognize. The goal is that all the students get the perfect score at the end of the OLETE which in turn assumes that they all have achieved mastery of the learning objectives using the test and/or exercise as the evaluation tool.

Instructions: For full learning, if you don't get a 100% score on the first take, please repeat taking the OLETE until you get a perfect score. There is no limit to the number of times taking this OLETE. The gauge of full learning is to get a 100% score on this OLETE. This is the essence of OLETE. A Certificate of Commitment and Accomplishment will be issued to those with 100% score.

Learning objectives:

At the end of the session, the student must be able to:

· Define scoring.

· Define grading.

· Define points.

· Define raw scores.

· Give examples of translated scores from raw scores.

· Differentiate norm-referenced and criterion-referenced system of grading.

· Discuss the different methods on how to determine the pass or cut score.

· Discuss the limitations of scoring and grading systems in a medical school.

· Explain why medical schools should go for a criterion-referenced systems of scoring and grading.

· Discuss the limitations of scoring and grading systems in medical schools.

· Discuss the usefulness of item analysis.

NAME (First - Middle - Surname) *

Age *

Sex *

Male

Female

Date OLETE Taken (Month - Day - Year) *

Email Address *

INSTRUCTIONS ON TYPES OF QUESTIONS WITH POINTS

Basically, two types: MULTIPLE CHOICE questions (with circles) - only one answer. CHECKBOX questions (with square boxes) - may have more than one correct choices [to get a full point, all correct choices must be ticked; missing one leads to WRONG answer and therefore, no point.] There may be matching questions and other types of objective questions.

ROJOSON EDUCATION FOR HEALTH DEVELOPMENT IN THE PHILIPPINES

Which of the following statements are valid regarding scoring and grading management systems?

1 point

SCORING AND GRADING MANAGEMENT SYSTEMS are 2 distinct but interrelated and should be integrated systems.

They should be implemented in all types of tools, such as in test-based, observation-based, and project-based tools.

Scoring and grading systems will be dependent on the type of educational measuring tools to be used.

Scoring Management System refers to a set of policies, processes and procedures on scoring of educational measuring tools.

Grading Management System refers to a set of policies, processes and procedures on grading of educational measuring tools.

Required

_______________ refers to the process of assigning numerical units to represent points that students will get and/or have gotten in educational measuring tools such as test-based, observation-based and project-based tools.

1 point

Scoring

Grading

_______________ refers to the process of applying standardized measurements for varying levels of achievements in a learning activity or course of study using the scores the students will get and/or have gotten in educational measuring tools such as test-based, observation-based and project-based tools.

1 point

Scoring

Grading

In terms of implementation of actual processes, there is a proper sequence: grading precedes scoring.

1 point

True

False

in the formulation of the plan for scoring and grading, there is already integration of the two processes – how to score and how to grade are already placed in the plan.

1 point

True

False

Terms in scoring and grading are often used interchangeably and at times, differently. Which of the following statements on their usages are valid?

1 point

POINTS = MARKS

SCORES MAY BE THE SAME AS POINTS BUT NOT ALL THE TIME.

TOTAL POINTS = SCORES

RATINGS USED IN BROADER SENSE THAN GRADES.

GRADES UTILIZE THE SCORES FOR DECISION-MAKING.

SCORES ARE CONVERTED TO GRADES.

Required

On POINTS, which of the following statements are valid?

1 point

Points are numeric units which represent relative value or weight of importance determined by a teacher on specific items in educational measuring tools such as tests, observation technique and project-based technique.

The total point gotten by the students in each educational measuring tool is the sum of all points assigned to each item of assessment.

The total point gotten by the students is the sum of all points on correct answers and sum of all numeric values in a rating scale.

Required

On GRADES, which of the following statements are valid?

1 point

Grades can be assigned as letters (usually A through F), as a range (for example, 1 to 5), as a percentage, or as a number out of a possible total (often out of 100).

Grades in medical schools are usually in numeric units.

Grades may be using raw scores or translated scores.

May be using norm-referenced or criterion-referenced systems for the passing grades.

Required

_______________ refers to the scores without any sort of adjustment or transformation such as simple number of questions answered correctly

1 point

Raw Scores

Translated Scores

_______________ refers to the scores resulting from some transformation(s) applied to the raw scores, adjusted and converted to a standardized grading system.

1 point

Raw Scores

Translated Scores

Match the statements in the Row with the types of referencing systems in scoring and grading. *

2 points

Norm-referenced System

Criterion-referenced System

Referencing how the score of a student compares to other students. (grading on a curve using mean and standard deviations)

Referencing how the score compares to a criterion such as a cut score or a body of knowledge. (grading using a set of defined standards)

What are the various acceptable terms used for "score at which a student passes or fails?" *

1 point

Pass or passing score

Cut score

Minimum pass level score

Required

The scoring and grading systems for selective objective tests are different from oral / essay tests. *

1 point

True

False

The scoring and grading systems for oral / essay tests and observation-based assessment tools are basically the same. *

1 point

True

False

The scoring and grading systems for observation-based and project-based assessment tools are basically the same. *

1 point

True

False

The medical school should come out with policies and procedures on its scoring and grading systems.

1 point

True

False

The students should be informed of these policies and procedures so that there will be NO misunderstanding.

1 point

True

False

What are the recommended practices of determining the pass score in medical schools? *

1 point

The passing score for an educational measuring tool should be set in accordance with the purpose of the exam (formative, summative, knowledge, skills, attitude, etc.)

The passing score set for an educational measuring tool must be defensible, acceptable, and credible (not arbitrary).

The process of determining an effective and valid cut score for an educational measuring tool is referred to as standard setting.

This requires a properly selected panel of expert judges and a suitable standard setting method, which best fits the purpose of the educational measuring tool.

Criterion-referenced system is recommended rather than norm-referenced system.

Required

What are the different methods of standard setting in criterion-referenced system of scoring and grading? *

1 point

Institution’s policy on absolute pass score such as 70% or 75%.

Nedelsky Method

Angoff Method

Ebel Method

Required

In the Angoff method, the experts assess each question and determine the percentage of 100 minimally competent students who will be able to answer the item correctly and then the minimum pass level is computed.

1 point

True

False

Shown a rubric of an educational measuring tool, determine whether it uses norm-referenced or criterion-referencing system.

1 point

Norm-referenced scoring and grading

Criterion-referenced scoring and grading

Shown a rubric of an educational measuring tool, determine whether it uses norm-referenced or criterion-referencing system.

1 point

Norm-referenced scoring and grading

Criterion-referenced scoring and grading

Regarding item analysis, which of the following statements are valid? *

1 point

One specific activity recommended in the continual evaluation and review of the scoring and grading systems is the so-called ITEM ANALYSIS.

Item analysis is a process which examines student responses to individual test items (questions) in order to assess the quality of those items and of the test as a whole.

Item analysis typically focuses on four major pieces of information: item difficulty, item discrimination, item distractors and response frequency.

Item analysis is especially valuable in improving items which will be used again in later tests, but it can also be used to eliminate ambiguous or misleading items in a single test administration.

item analysis is valuable for increasing instructors’ skills in test construction and identifying specific areas of course content which need greater emphasis or clarity.

Required

What are ROJoson's thoughts, perceptions, opinions and recommendations on Scoring and Grading of Educational Measuring Tools for Medical Students’ Learning and Performance? *

1 point

There is a need to measure the learning and performance of medical students after the course of study and thereafter, to decide whether or not to certify them as graduates who are qualified to practice medicine in the Philippines (that is, after passing the Philippines Physician Licensure Exam).

The educational measuring tools for medical students’ learning and performance are not perfect and probably will never be perfect, whatever they are.

Despite this imperfection, they should still be administered because there is a need to measure, to evaluate and to determine who are eligible to be basic-generalist physicians.

The tools should be CREDIBLE and VALID.

They should not ARBITRARY and not using the norm-referenced scoring and grading (using mean or average as a passing score, so called grade on a curve).

Criterion-referenced scoring and grading must be used.

Even the criterion-referenced scoring and grading are not perfect. But at least, students are assessed and evaluated against a set of criteria or standards on mastery of the professional contents of the medical curriculum.

There is no “gold standard” for setting passing scores, in the standard setting in the criterion-referenced system.

Usually the choice of the standard setting method is based on the available resources and the practical realities of the educational environment. As such, it is critical to document all procedures used in establishing the passing standard, especially in a language the school is willing to make public.

Every effort should be exerted to use one of the standard setting approaches described in the literature so as to make the passing scores defensible and meaningful.

Also important is the continual improvement of the scoring and grading systems. There should be evaluation and review of the systems at planned intervals.

One specific activity recommended in the continual evaluation and review of the scoring and grading systems is the so-called ITEM ANALYSIS.

When considering a student’s performance on a summative exam, educators should take into account the body of evidence that student has built up during the duration of the academic career.

Required

END OF ROJ-OCIL-OLETE for Tools in Measuring Program and Learning Outcomes of Medical Students – A Guide in Scoring and Grading (Part 3)

Learning objectives covered in this OLETE:

At the end of the session, the student must be able to:

· Define scoring.

· Define grading.

· Define points.

· Define raw scores.

· Give examples of translated scores from raw scores.

· Differentiate norm-referenced and criterion-referenced system of grading.

· Discuss the different methods on how to determine the pass or cut score.

· Discuss the limitations of scoring and grading systems in a medical school.

· Explain why medical schools should go for a criterion-referenced systems of scoring and grading.

· Discuss the limitations of scoring and grading systems in medical schools.

· Discuss the usefulness of item analysis.

What is your before and current state of knowledge on the Tools in Measuring Program and Learning Outcomes of Medical Students – A Guide in Scoring and Grading (Part 3) (Don't be shy in answering this - it will be kept confidential.)
*

Minimal amount of knowledge

Moderate amount of knowledge

Great amount of knowledge

Before the ROJ-OCIL- OLETE

After the ROJ-OCIL- OLETE

Feedback, comments and suggestions on ROJ-OCIL-OLETE on Tools in Measuring Program and Learning Outcomes of Medical Students – A Guide in Scoring and Grading (Part 3) (Although there is no point assigned to this question, it is one of the requirements to pass the OLETE. Pls. do not put NONE or NA - Not Applicable. Thank you.)
*

Template of a Certificate of Commitment and Accomplishment - for those with a 100% score in this OLETE.

Thank you for taking and passing this OLETE. Once you pass this OLETE, email (rjoson2001@yahoo.com) and text (09188040304) for me to prepare and send your certificate.

Submit

Clear form

Never submit passwords through Google Forms.

This content is neither created nor endorsed by Google. Report Abuse - Terms of Service - Privacy Policy

Forms