How to design The Objective Structured Clinical Examination (OSCE) in hand surgery

Examinations are formidable even to the best prepared, for the greatest fool may ask more than the wisest man can answer. (Charles Colton, 1780-1832)

The OSCE has been in place since 1970 and used in Dundee and now widely employed in medical education as a cost effective tool for competence assessment. The literature’s view on this is well summarised as follows (Walther N.K.A. van Mook, 2009) :-

• The ‘magic bullet’ regarding professionalism assessment does not exist.

•        Current ‘state-of-the-art’ in the assessment of professional behaviour is combining multiple methods (triangulation) in varied contexts with multiple observers over a longitudinal trajectory.

•        Current state of the art in education of professional behaviour is assessment with adequate follow-up and with the provision of opportunities for remediation; only assessment without remediation loses ground.

OSCE for fixation in an oblique fracture of the 3^rd metacarpus in a plastic bone model

Aim:

To assess the candidates ability to perform inter fragmentary compression of a fracture of a metacarpus with a 2 mm screw using the lag screw principle.

This OSCE station will test a candidate in the following domains:-

Knowledge: The principles of lag screw fixation

Skills: The ability to accurately and effectively place a lag screw to produce adequate compression across a standardised oblique fracture in a plastic bone model.

Attitude: Attention to detail and methodical planning process.

The Station layout  - on a standard student desk 

Scenario

A standardise oblique fracture created with a saw in the 3 metacarpus of a plastic hand model

Instructions for candidate

1. Stably fix the fracture that has been reproduced in this model using the implants provided.
2. Chose your implant and place it accurately to obtain stable fixation.
3. Use the equipments provided to assist you.
4. Plan you procedure on the paper provided.
5. Use all protective equipment provided.
6. Ensure that you fixation is safe, stable and appropriate.

Instruction to examiner

The time for this station is 15 minutes.

Ensure the candidate chooses his fixation choice and plans it first on paper before proceeding.

1. Choose lag screw technique using a single 2 mm screw
2. Reduces accurately the fracture and holds it with a reduction clamp
3. Chooses the site for placement of the lag screw – right angle to fracture and at least  4 mm from the fracture edges
4. Uses a 2mm drill guide to drill the pilot hole with a 2 mm drill ensuring only the near cortex is drilled
5. Uses the 1.5mm drill sleeve in the pilot hole to drill a 1.5mm hole in the far cortex with a 1.5 mm drill
6. Measures accurately the length of the required screw
7. Countersinks the entry hole to ensure the head of screw will be flushed with the bone
8. Chooses the appropriate screw and confirms the length on the measuring scale
9. Inserts the screws and
10. Confirm compression at fracture site.

The resources required

1. Plastic Bones hand model
2. Cork board with rubber bands to anchor model
3. 2 mm Hand Compact Hand Set including instruments and implants (Synthes)
4. Power drill cordless

Mark sheet

A checklist marking has been used here as it allows for quantification of results and provides objectivity and by adding a heavy weight age to the choice of fixation it shows the importance of choosing the best options for fixation of an oblique fracture.

A global rating was not used as it did not allow for specific areas of assessment and the checklist allows for more accurate feedback for the candidates in areas of weakness for remedial learning.

                                                                                    Max                                         Actual

Ensure the candidate chooses the right fixation technique – a lag screw technique using a single 2 mm screw	4
Reduces accurately the fracture and holds it with a reduction clamp	1/2
Chooses the site for placement of the lag screw – right angle to fracture and at least  4 mm from the fracture edges	1/2
Uses a 2mm drill guide to drill the pilot hole with a 2 mm drill ensuring only the near cortex is drilled	1/2
Uses the 1.5mm drill sleeve in the pilot hole to drill a 1.5mm hole in the far cortex with a 1.5 mm drill	1/2
Measures accurately the length of the required screw	1/2
Countersinks the entry hole to ensure the head of screw will be flushed with the bone	1/2
Chooses the appropriate screw and confirms the length on the measuring scale	1/2
Inserts the screws in line with drill hole	1/2
Confirm compression at fracture site.	2

Max                                                                             10

Advantages of this OSCE

• Each candidate undergoes the same assessment material and scoring system under identical conditions

• Versatile format to assess any part of curriculum in hand surgery

• Potentially provides both candidate and trainer with detailed analysis of performance

• Potential for objective feedback

Disadvantages of this OSCE

• Resource intensive needing planning and resources like equipment, personnel and time

• Not a holistic assessment tool as this only test for psychomotor skills

• Examiners need to be trained to assess objectively

References

Walther N.K.A. van Mook et al

General considerations regarding assessment of professional behaviour, European Journal of Internal Medicine, Volume 20, Issue 4, July 2009, Pages e90-e9

Evidence of e-learning in surgical education -  literature review 

GoPro head worn camera recording for surgical education CSC, Cambodia

9 December 2012

Introduction

  

The database used to search for articles was Academic Search Complete from the digital library of the OUM LMS.  The search terms e-learning, medical education and surgery were used. This Boolean search returned 596 articles, of which those relevant articles with research on various e-learning techniques as the dependent variable, and outcomes in learning as the independent variables on the population of surgical trainees were chosen. The search was limited to full articles in peer-reviewed journals between   2002 and 2012.

One of  the  references   (Larvin, 2009), was used as a framework to understand the current thinking and practice of e-learning in the surgical environment. Larvin is an authority on e-learning in surgical education both as a leader and practitioner in the Royal Colleges of Surgeons in the United Kingdom, which is the body that is responsible for surgical education and training in the United Kingdom.  He acknowledges that e-learning remains a novelty in surgical education as compared to undergraduate medical education where it has become common practice. Larvin adds that e-learning has tremendous potential in surgical education as compared to other specialties. The introduction of the European Working Time Directive has added further challenges to surgical education in Europe. All the Royal Colleges of Surgeons in the United Kingdom have agreed on the importance of e-learning to augment asynchronous learning for surgical trainees. The results of the survey conducted by the Royal Colleges on the effectiveness of e-learning has shown great acceptance by surgical trainees and emphasises the importance of adding value, linking to existing and other e-learning resources. Formative assessment within e learning has been found to be valuable as it provides safe and valuable feedback for surgical trainees.(Larvin, 2009 p 136)

Methodology and measure

Various methodologies were employed in these researches to study the effect and acceptance of eLearning in surgical training. Citak et al., Corrigan et al., and Coughlan et al.,  (Citak et al., 2009; Corrigan et al., Coughlan et al., Coughlan et al 2012) used a similar  methodology of survey to assess the effectiveness of e-learning in surgical education.  Cital et al  (Citak et al., 2009) had used an online trauma surgery repository for the teaching of trauma surgery for medical students in Hanover. Here they surveyed the students after the use of the independent variable which was the online trauma surgery Schoolbook for the academic year and the dependent variable was the satisfaction obtained from the use of the e-learning trauma surgery repository by completing the questionnaire. The questionnaire designed for this had included the evaluation of the students of the e-learning based on the mode of accessing, utilization and opinion toward the e-learning material. .  Corrigan et al (Corrigan et al., 2012) similarly used an online repository of surgical educational material as the independent variable in their study and surveyed the participants on the effectiveness of the online repository through questionnaire.  Here the authors had use a commercial tool to measure usage of the online repository and a survey based feedback. Response rate for those participants was 67% making it valid. Effectiveness of the programme was measured with a focus group again having the same response rate. Coughlan et al   (Coughlan et al 2012) used a CD-ROM format of e-learning as the independent variable and  survey questionnaire to evaluate the surgical content and learning benefit among surgical trainees. By using this they excluded problems of web-based learning in terms of connectivity and bandwidth. They had modified a validated questionnaire which was used to evaluate instructional multimedia application interface features proposed by Reeves and Harmon (1994).

Bhatti et al.,  Hull et al and Ridgway et al., however performed primary research by using various e-learning programs as the independent variable and measured performance, using standardise assessment tools, as the dependent variable. Bhatti et al (Bhatti et al., 2011) had used standard lecture and an e-learning program as their independent variables and measure its effectiveness using an assessment tool consisting of Multiple-choice questions and Extended matching questions  which are validated knowledge acquisition assessment tools in medical education, where performance in this assessment was the dependent variable. Hull et al (Hull et al., 2009) had used bedside teaching and computer-based learning as the  independent variable in their study however they included sequencing of these as another independent variable in their study.  The effectiveness in producing learning in their study was performed by measuring performance at a standardise validated assessment tool in the clinical setting, the objective structured clinical examination (OSCE).

Ridgway (Ridgway et al., 2007) studied the effect of two  e-learning programs, independent variables, one being a text only learning material and the other text and  audio . The effectiveness of these independent variables was assessed by the use of standard assessment tool which was a multiple-choice questionnaire  designed   specifically to assess learning of  the text and graphical material presented in the lecture

series.  Acceptability was assessed by standardized anonymous questionnaire based on Likert scales for the different independent variables and focusing on ease of use, accessibility, relevance, content and satisfaction measure using survey questionnaire.

Population

Except for the Coughlan study where the population were surgical trainees all the other studies  included medical students in the population.  In Bhatti et al., study(Bhatti et al., 2011) , the population of medical student were subjected to various learning materials  as the independent variable, they being normal lecture and e-learning and were randomised using a web-based randomisation program called QuickCalcs which has been validated as an appropriate tool in the clinical setting by Suresh et al (Suresh, 2011).

In the Hull et al study the medical student population was randomised using a randomisation software into two groups where one had the sequencing of bedside teaching followed by computer-based learning as compared to the other which had computer-based learning prior to bedside teaching as the independent variable.

In the Ridgway et al study there was no randomisation of the population of medical students. All of them were entered into the independent variables which consist of a lecture series consisting of text with no audio and another  independent variable consisting of lectures with audio. Therefore the whole populations were subjected to different intervention in the form of the learning material being a multimedia e-learning and without multimedia.

In the other three studies (Citak et al., 2009; Corrigan et al., Coughlan et al., 2012) there was  no randomisation of the population as it was a survey of all those involved and  the return rate of questionnaires were satisfactorily and acceptable levels for qualitative research.

Conclusion

The results from the survey type studies in this literature review (Citak et al., 2009; Corrigan et al., Coughlan et al 2012), showed general acceptance and utilisation of e-learning by students in surgical education at undergraduate and postgraduate level. Citak et al had shown that even the instructors had found the e-learning useful in their teaching. Coughlan et al had shown the participants had benefited and found the e-learning relevant to the learning needs and indicated that the e-learning be expanded to other nonsurgical topics.

The experimental papers, Bhatti et al.,  Hull et al and Ridgway et al , the authors had shown that e-learning with multimedia content is effective in bringing about knowledge acquisition in undergraduate teaching. Bhatti et al had shown that “e Learning supplemented with a podcast results in greater knowledge acquisition when compared with a traditional lecture, without a loss of satisfaction with teaching.”(Bhatti et al., 2011 p 459). Hull et al in their study showed that e-learning with bedsides teaching in that sequence was most effective for students to learn  orthopaedic knee examination however if only one modality was available bedside teaching was preferred than solitary e-learning. Ridgway et al clearly showed that the addition of audio to web-based learning improved examination results.(Ridgway et al., 2007 p 168).

This literature reviews has shown that there is evidence to show the effectiveness and usefulness of e-learning programs in surgical education. It has good acceptance and utilisation of e-learning repositories by learners in surgery and produces some guidelines for design of instructional materials for e-learning in the surgical educational environment. 

References

Bhatti, I., Jones, K., Richardson, L., Foreman, D., Lund, J., & Tierney, G. (2011). E-learning vs lecture: which is the best approach to surgical teaching? Colorectal Disease, 13(4), 459–462.

Citak, M., Calafi, A., Kendoff, D., Kupka, T., Haasper, C., Behrends, M., Krettek, C., et al. (2009). An internet based learning tool in orthopaedic surgery: Preliminary experiences and results. Technology & Health Care, 17(2), 141–148.

Corrigan, M., McHugh, S., Sheikh, A., Lehane, E., Shields, C., Redmond, P., Kerin, M., et al. (2012). Surgent University: the establishment and evaluation of a national online clinical teaching repository for surgical trainees and students. Surgical innovation, 19(2), 200–204. 

Coughlan, J., and Brinkman, W.-P. (2011). Design considerations for delivering e-learning to surgical trainees.  International journal of e-health and medical communications . 2(2). 14-23

 Hull, P., Chaudry, A., Prasthofer, A., & Pattison, G. (2009). Optimal sequencing of bedside teaching and computer-based learning: a randomised trial. Medical education, 43(2), 108–112. 

Larvin, M. (2009). E-Learning in surgical education and training. ANZ Journal of Surgery, 79(3), 133–137.

Reeves, T. C., &  Harmon, S. W. (1994). Systematic evaluation procedures for interactive multimedia for education and training. In S. Reisman (Ed.), Multimedia Computing - preparing for the 21st Century (pp. 472-505). London: Idea Group Publishing

Ridgway, P. F., Sheikh, A., Sweeney, K. J., Evoy, D., McDermott, E., Felle, P., Hill, A. D., et al. (2007). Surgical e-learning: validation of multimedia web-based lectures. Medical Education, 41(2), 168–172.

Suresh, K. (2011). An overview of randomization techniques: An unbiased assessment of outcome in clinical research. Journal of Human Reproductive Sciences, 4(1), 8–11. doi:10.4103/0974-1208.82352

Evaluating MOOC - Massive open online course - the future of education

   

Massive open online course (MOOC) are the product of the newer theories of networked learning and connectivitism. It allows for online courses targeting a large audience with large scale interactive participation and open access (free access) via the web. It provide interactive user forums that help build a community for the students and teachers. (“Massive open online course” 2013).

Though massive, many of these courses do not have that many registrants and few complete the course. The activities could be synchronous or asynchronous, and is usually a flexible structure allowing for self-paced on demand learning. It allows for universally accessibility for a course thereby enabling extensive collaborative and interactive opportunities for students. The main disadvantage is that they have low teacher interactivity and feedback and this will not be acceptable for some students

who expect or thrive on a high level of teacher interaction. This phenomenon has revolutionised the relationship between learner and instructor and between schools and the wider community (Thompson, K November 2011).

The social pedagogical theoretical frameworks involved in the authors view are mainly networked learning and connectivitism.

The basic design of MOOC requires instructional design that facilitates large-scale feedback and interaction. It includes (“Massive open online course” 2013) :

·        Crowd-sourced interaction and feedback by leveraging the MOOC network, e.g. for peer-review, group collaboration

·        Automated feedback through objective, online assessments, e.g. quizzes and exams

The principles of connectivist pedagogy of MOOC include (“Massive open online course” 2013):

·       Aggregation.   It allows for a massive amount of content to be produced anywhere online, which is later aggregated as content page to participants on a regular basis. 

·        Remixing, - associating materials created within the course with each other and with materials elsewhere.

·       Re-purposing of aggregated and remixed materials to suit the goals of each participant.

·       Feeding forward, sharing of re-purposed ideas and content with other participants and the rest of the world.

12 benefits of a MOOC (Tangient LLC.( Jul 7, 2011)

• 1.     You can organize a MOOC in any setting that has connectivity (which can include the Web, but also local connections via Wi-Fi e.g.)
• 2.     You can organize it in any language you like (taking into account the main language of your target audience)
• 3.     You can use any online tools that are relevant to your target region or that are already being used by the participants
• 4.     You can move beyond time zones and physical boundaries
• 5.     It can be organized as quickly as you can inform the participants (which makes it a powerful format for priority learning in e.g. aid relief)
• 6.     Contextualized content can be shared by all
• 7.     Learning happens in a more informal setting, at a place of your convenience and often around your own schedule.
• 8.     Learning can also happen incidentally thanks to the unknown knowledge that pops up as the course participants start to exchange notes on the course’s study
• 9.     You can connect across disciplines and corporate/institutional walls
• 10. You don’t need a degree to follow the course, only the willingness to learn (at high speed)
• 11. You add to your own personal learning environment and/or network by participating in a MOOC
• 12. You will improve your lifelong learning skills, for participating in a MOOC forces you to think about your own learning and knowledge absorption

Example of a MOOC Course

Platform: Udemy course

Title: Foundations of Business Strategy by Michael Lenox

Site: https://www.udemy.com/foundations-of-business-strategy/

The Udemy platform allows for each course to have a repository of videos, PowerPoint presentations, PDFs, documents, articles, links, pictures, and live sessions all formed into a series of lessons.

The user friendly interface allows for quick and easy organization and editing of lessons. It has the facility for synchronous teaching using virtual learning environment with live video chats, chat room, white boarding and presentation sharing.

A well-planned course balances three levels of interaction: faculty-to-student, student-to-student, and student-to-resources.(Boettcher, J. V. (2007).

Analyses of this MOOC site was performed using the principles governing connectivity that was included by Siemens G (2005).

Learning and knowledge rests in diversity of opinions.

In this category this course did not provide an opportunity for diversity of opinions as there was no connection with other nodes and other sources of information and knowledge.

Learning is a process of connecting specialized nodes or information sources.

Here too, this course did not have connection to specialised nodes or information sources. The resources were confined to the course material that was available by the author and teacher with no other source material being made available

Learning may reside in non-human appliances.

Here this course did deliver in that it was a repository of learning artefacts in the form of video and PowerPoint slides

Learning is more critical than knowing.

The format of this course was mainly one of knowing rather than learning as there were no connectivity to other nodes and other sources of information to see the whole process of learning among the learners and teachers and other repository of resources.

Maintaining and nurturing connections

The course had chat functions but this was mainly for asynchronous use and there was no opportunity for synchronous communication between learners.

Perceiving connections between fields, ideas and concepts.

There was no connection between the various aspects of learning within this course. It was more for one way of information being delivered to the learner with very little connectivity and any form of facilitation of connection between learners. There were no forums or wikis to allow for knowledge to be shared and built upon for consumption among the learners.

Currency (accurate, up-to-date knowledge)  

This was clearly lacking in this course due to the fact that most of the material was outdated and there were no new information being added on either by the teacher or any of the learners. There was very little opportunity for learners to contribute to the learning of others.

Decision-making as a learning process.  

This component was not present in the course and very little flexibility was allowed for the participants.

Assessing the course for the enabling conditions as outlined by Pettenati  e al (Pettenati, M. C., & Cigognini, M. E. 2007), the following were noted for each of the areas.

The ability of the learners to have basics technology skill

By being an online course this was self-selecting as those who do not have basic technological skills would not have been able to register and sign up or navigate within the course

Generation and support of motivation

The course was unable to provide any form of support or motivation as it was purely the delivery of content material with very little support from the teachers or peer support through forums.

Relevance in learning and perceived real advantage by the learner

The content of the course was relevant with the availability and sequencing of topics that were relevant to management and strategy however it did not have a clearly stated learning outcome.

Positive group membership and culture

There were no opportunities for peer learners or tutors to connect with each other and develop and build the community with membership and culture. The only interaction that was available in the hosting platform was chat function.

 

 Social climate allows for mutual understanding and social grounding

There was no opportunity for peer-to-peer connection in this course and there was no possibility of social group being formed with mutual understanding and social grounding  

References:

  

Boettcher, J. V. (2007).Citing website in Ten core principles for designing effective learning environments: Insights from brain research and pedagogical theory. Retrieved April 10 2913 from http

Goodyear, P.(2001) . Effective networked learning in higher education: notes and guidance. Publication of JCALT 2001.

 Massive open online course (n.d.). In Wikipedia. Retrieved March 11, 2013, from http://en.wikipedia.org/wiki/ Massive_open_online_course

Pettenati, M. C., & Cigognini, M. E. (2007). Social Networking Theories and Tools to Support Connectivist Learning Activities. International Journal of Web-Based Learning and Teaching Technologies (IJWLTT), 2(3), 42-60.

Siemens, G. (January 2005). Citing Website in International Journal of Instructional Technology and Distance Learning Retrieved March 21 2013,:From http://www.itdl.org/Journal/Jan_05/article01.htm

Tangient LLC.( Jul 7, 2011) Citing website MoocGuide Benefits and challenges of a MOOC.. Retrieved april 10, 2013.from http://moocguide.wikispaces.com/2.+Benefits+and+challenges+of+a+MOOC

Thompson, K (November 2011). Citing Website. In 7 things you should know about MOOCS, . Retrieved April 4, 2013, from http://net.educause.edu/ir/library/pdf/ELI7078.pdf.