SUBMITTED BACKGROUND PAPERS


Some submitted papers relate to the work of more than one Theme Group. These are listed below in the "General" category. Papers that mainly relate to one specific theme are listed separately below. Where the submitter has not sent the full text, the reference to the paper is given, together with a link to the relevant website where the full text may be acquired.

General

Title: Tertiary Assessment & Higher Education: Student Outcomes: Policy, Practice and Research
Authors: Victoria University of Wellington, New Zealand
Submitted by:
Zubair Amin
Date submitted: August 2009

Title:
University of Auckland Teaching and Learning Policy: Assessment of Student Learning
Submitted by: Zubair Amin
Date submitted:
August 2009

Title:
Development of the Objective Structured System-Interaction Examination
Authors: Susan Hingle, Richard B Rosher, Sherry Robinson, Nancy McCann-Stone, Christine Todd, Michael Clark
Submitted by: Susan Hingle
Date submitted: October 2009

Title:
Description of a Developmental Criterion-Referenced Assessment for Promoting Competence in Internal Medicine Residents
Authors: Andrew Varney, Christine Todd, Susan Hingle, Michael Clark
Submitted by: Susan Hingle
Date submitted: October 2009

Title:
The educational effects of portfolios on undergraduate student learning: a BEME systematic review (BEME Guide No 11)
Authors: Sharon Buckley et al
Submitted by: Pat Lilley
Date submitted: October 2009

Title:
The effectiveness of portfolios for post-graduate assessment and education: BEME Guide No 12)
Authors: Claire Tochel et al
Submitted by: Pat Lilley
Date submitted: October 2009

Title: Centralized Assessment in Graduate Medical Education: Cents and Sensibilities
Authors: Dianne Wagner and Monica L Lypson
Submitted by: Dianne Wagner
Date submitted: November 2009

Title: The Dreyfus Model of Clinical Problem-Solving Skills Acquisition: A Critical Perspective
Author: Adolfo Peña
Submitted by: Adolfo Peña
Date submitted: December 2009
 
Title:
Educative Assessment in 2010 and Beyond
Authors: Alice Fornari, David Battinelli and Lawrence Smith
Submitted by: Alice Fornari
Date submitted: December 2009
 
Title:
Thin Slices of Negotiation: Predicting Outcomes From Conversational Dynamics Within the First 5 Minutes
Authors: Jared F Curhan and Alex Pentland
Submitted by:
Date submitted: December 2009
 
Title:
An Analysis of Long-Term Outcomes of the Impact of Curriculum: A Comparison of the Three- and Four- Year Medical School Curricula
Authors: Jocelyn M Lockyer, Claudio Violato, Bruce J Wright and Herta M Fidler
Submitted by: Jocelyn Lockyer
Date submitted: December 2009
 
Title:
Charting the Road to Competence: Developmental Milestones for Internal Medicine Residency Training
Authors: Michael L Green, Eva M Aagaard, Kelly J Caverzagie, Davoren A Chick, Eric Holmboe, Gregory Kane, Cynthia D Smith and William Iobst
Submitted by: Michael L Green
Date submitted: December 2009
 
Title:
Defining competence for the purposes of certification by the College of Family Physicians of Canada: The new evaluation objectives in family medicine
Authors: College of Family Physicians of Canada
Submitted by:
Date submitted: December 2009
 
Title:
Developing Generalisable Measures of Knowledge and Skill Outcomes in Higher Education
Author: Hamish Coates
Submitted by: Hamish Coates
Date submitted: December 2009
 
Title:
Portfolios as a method of student assessment: AMEE Guide No 24
Authors: Miriam Friedman Ben-David, M H Davis, R M Harden, P W Howie, J Ker and M J Pippard
Submitted by: Pat Lilley
Date submitted: December 2009
 
Title:
The assessment of learning outcomes: AMEE Guide No 25
Authors: James M Shumway and Ronald M Harden
Submitted by: Pat Lilley
Date submitted: December 2009
 
Title:
Assessing reproductive endocrinology competencies among senior medical students in the European Union: deployment of a new clinical module at the Royal College of Surgeons in Ireland
Author: Eric Scott Sills
Submitted by: Eric Scott Sills
Date submitted: January 2010

Title: Assessment of medical competence using an objective structured clinical examination (OSCE). ASME Medical Education Booklet no 8
Authors: R M Harden and F A Gleeson
Reference: Association for the Study of Medical Education, 1979
Submitted by: R M Harden
Date submitted: April 2010

Title: Assessment of Clinical Competence using Objective Structured Examination
Authors: R McG Harden, Mary Stevenson, W Wilson Downie, G M Wilson
Reference: Brit Med J 1975, 1, 447
Submitted by: R M Harden
Date submitted: April 2010

Title: Quantifying Construct Validity: Two Simple Measures
Authors: Drew Westen, Robert Rosenthal
Reference: Journal of Personality and Social Psychology, 2003, 84, 3, 608-618
Submitted by: R M Harden
Date submitted: April 2010

Title: Validity: on the meaningful interpretation of assessment data
Author: Steven M Downing
Reference: Medical Education 2003;37:830-837
Submitted by: R M Harden
Date submitted: April 2010

Title: Setting standards on educational tests
Author: John J Norcini
Reference: Medical Education 2003;37:464-469
Submitted by: R M Harden
Date submitted: April 2010

Title: Can there be reliability without "reliability?"
Author: Robert J Mislevy
Reference: Journal of Educational and Behavioral Statistics, 2004, 29, 2, 241-244
Submitted by: R M Harden
Date submitted: April 2010

Title: Achieving acceptable reliability in oral examinations: an analysis of the Royal College of General Practitioners membership examination's oral component
Authors: Val Wass, Richard Wakeford, Roger Neighbour, Cees van der Vleuten
Reference: Medical Education 2003;37:126-131
Submitted by: R M Harden
Date submitted: April 2010

Title: Portfolio assessment in medical students' final examinations
Authors: M H Davis, M Friedman Ben-David, R M Harden, P Howie, J Ker, C McGhee, M J Pippard, D Snadden
Reference: Medical Teacher 2001, 23, 4, 357-366
Submitted by: R M Harden
Date submitted: April 2010

Title: Assessing clinical competence: A pilot project to evaluate the feasibility of a standardized patient-based practical examination as part of the Swiss certification process
Authors: NV Vu et al
Reference: Swiss Med Weekly 2006, 136, 392-99
Date submitted: May 2010


Criteria for a good assessment

Title:
Setting Performance Standards on Educational Assessments and Criteria for Evaluating the Process
Author: Ronald K Hambleton
Submitted by: John Norcini
Date submitted: November 2009

Title:
Standard Setting in Student Assessment: AMEE Guide No 18
Author: Miriam Friedman Ben-David
Submitted by: Pat Lilley
Date submitted: December 2009


Technology-based Assessment

Title:
The use of a problem-specific mobile electronic clinical evaluation tool (eCEX) in the assessment of medical student’s clinical competencies: A pilot study
Authors: Gary Ferenchick, Steve Sneed, Asad Mohmand, David Solomon
Submitted by: Gary Ferenchick
Date submitted: September 2009


Performance Assessment

Title:
Review of work-based assessment methods
Author: University of Sydney, Centre for Innovation in Professional Health Education & Research
Submitted by: John Norcini
Date submitted: November 2009

Title:
Unannounced Standardized Patients: A Gold Standard for the Research and Practice of Clinical Performance Assessment
Authors: Alan Schwartz and Saul Weiner
Submitted by: Alan Schwartz
Date submitted: December 2009

Title:
Cognitive, Social and Environmental Sources of Bias in Clinical Performance Ratings
Authors: Reed G Williams, Debra A Klamen and William C McGaghie
Submitted by: Debra Klamen
Date submitted: December 2009
 
Title:
The Use of Real patients, Simulated Patients and Simulators in Clinical Examinations: AMEE Guide No 13
Authors: J P Collins and R M Harden
Submitted by: Pat Lilley
Date submitted: December 2009

Title:
Remediating Students’ Failed OSCE Performances at One School: The Effects of Self-Assessment, Reflection, and Feedback
Authors: Casey B White, Paula T Ross and Larry D Gruppen
Submitted by: Casey White
Date submitted: December 2009

Title:  Assessment of clinical competence using an objective structured clinical examination (OSCE)
Authors: Harden RM, Gleeson FA.
Reference: Med Educ 1979; 13: 41–54
Submitted by: Kathy Boursicot
Date submitted: January 2010 

Title: Techniques for measuring clinical competence: objective structured clinical examinations
Authors: Newble DI.
Reference: Med Educ 2004; 38: 199–203
Submitted by: Kathy Boursicot
Date submitted: January 2010

Title: OSCEs and other structured assessments of clinical competence
Authors: Katharine AM Boursicot, Trudie E Roberts, William P Burdick,
Reference: Understanding Medical Education Series Booklet, ASME, Edinburgh, 2007
Submitted by: Kathy Boursicot
Date submitted: January 2010

Title: Workplace-based assessment in clinical training
Author: Norcini JJ.
Reference: Understanding Medical Education series Booklet, ASME, Edinburgh, 2007
Submitted by: Kathy Boursicot
Date submitted: January 2010

Title: Performance based assessment: lessons learned from health professions
Authors: Swanson DB, Norman GR, Linn RL
Reference: Educational Researcher 1995; 24(5):5-11
Submitted by: Kathy Boursicot
Date submitted: January 2010

Title: The Objective Structured Clinical Examination. The new gold standard for evaluating postgraduate clinical performance
Authors: Sloan DA, Donnelly MB, Schwartz RW, Strodel WE
Reference: Ann Surg 1995; 222(6):735-742
Submitted by: Kathy Boursicot
Date submitted: January 2010

Title: The relationship between competence and performance: implications for assessing practice performance
Authors: J-J Rethans, J J Norcini, M Baron-Maldonado, D Blackmore, B C Jolly, T LaDuca, S Lew, G G Page, L H Southgate
Reference: Med Educ 2002;36;901-909
Submitted by: Kathy Boursicot
Date submitted: January 2010

Title: Large-scale High-stakes Testing with an OSCE: Report from the Medical Council of Canada
Authors: Richard K Reznick, David Blackmore, W Dale Dauphinee, Arthur I Rothman, Sydney Smee
Reference: Acad Med 1996, 71, 1, ppS19-S21
Submitted by: Kathy Boursicot
Date submitted: January 2010

Title: ABC of learning and teaching in medicine: Work based assessment
Author: John J Norcini
Reference: BMJ 2003, 326, pp 753-755
Submitted by: Kathy Boursicot
Date submitted: January 2010

Title: The Mini-CEX: A Method for Assessing Clinical Skills
Authors: John J Norcini, Linda L Blank, F Daniel Duffy, Gregory S Fortna
Reference: Annals of Internal Medicine 2003, 136,6, pp 478-483
Submitted by: Kathy Boursicot
Date submitted: January 2010

Title: Analytic global OSCE ratings are sensitive to level of training
Authors: Brian Hodges, Jodi Herold McIlroy
Reference: Med Educ 2003;37:1012-1016
Submitted by: Kathy Boursicot
Date submitted: January 2010

Title: Defensible Assessment of the Competency of the Practicing Physician
Authors: John P W Cunnington, Eileen Hanna, John Turnbull, Tiina B Kaigas, Geoffrey R Norman
Reference: Acad Med 1997, 72, pp 9-12
Submitted by: Kathy Boursicot
Date submitted: January 2010

Title: Implementing workplace-based assessment across the medical specialties in the United Kingdom
Authors: James R Wilkinson, James G M Crossley, Andrew Wragg, Peter Mills, George Cowan, Winnie Wade
Reference: Med Educ 2008;42:364-373
Submitted by: Kathy Boursicot
Date submitted: February 2010

Title: Initial evaluation of the first year of the Foundation Assessment Programme
Authors: Helena Davies, Julian Archer, Lesley Southgate, John Norcini
Reference: Med Educ 2009;43:74-81
Submitted by: Kathy Boursicot
Date submitted: February 2010

Title: Assessor training: its effects on criterion-based assessment in a medical context
Authors: Godfrey Pell, Matthew S Homer, Trudie E Roberts
Reference: Int J of Res & Meth in Educ 2008;31:143-154
Submitted by: Kathy Boursicot
Date submitted: February 2010

Title: The reliability of workplace-based assessment in postgraduate medical education and training: a national evaluation in general practice in the United Kingdom
Authors: Douglas J Murphy, David A Bruce, Stewart W Mercer, Kevin W Eva
Reference: Adv in Health Sci Educ 2009;14:219-232
Submitted by: Kathy Boursicot
Date submitted: February 2010



Assessment of Professionalism

Title: Developing an outcomes-based charter for medical professionalism to direct teaching and assessment - a South African Experience
Authors: R Delport, C Krüger, M van Rooyen, R R du Preez, G Pickworth, I van Huyssteen, I Treadwell, M Kruger
Submitted by: R Delport
Date submitted: December 2009

Title: Assessing professional behavior: yesterday, today, and tomorrow.
Authors: Arnold L.
Reference: (2002) Acad Med 77(6):502-15. (PDF)

Title: Can There Be a Single System for Peer Assessment of Professionalism among Medical Students? A Multi-Institutional Study.
Authors: Arnold L, Shue CK, Kalishman S, Prislin M, Pohl C, Pohl H, Stern DT.
Reference: (2007) Academic Medicine 82(6), 578-586. (PDF)

Title: Professionalism in medicine: results of a national survey of physicians. 
Authors: Campbell EG, Regan S, Gruen RL, Ferris TG, Rao SR, Cleary PD, Blumenthal D.
Reference: (2007)  Ann Intern Med. 147(11):795-802. (PDF)

Title: Expectations and obligations: Professionalism and medicine's social contract with society
Authors: Cruess RL, Cruess SR.
Reference: Perspect Biol Med. 2008; 51(4):579-598.

Title: The professionalism mini-evaluation exercise: A preliminary investigation.
Authors: Cruess R, McIlroy J, Cruess S, Ginsburg S, Steinert Y.
Reference: Acad Med. 2006; 81(10 Suppl):S74-8.

Title: Context, conflict, and resolutions: A new conceptual framework for evaluating professionalism.
Authors: Ginsburg S, Regehr G, Hatala R, McNaughton N, Frohna A, Hodges B, Lingard L, and Stern D.
Reference:  Acad. Med. 2000; 75: (10 suppl); 96-511. (PDF)

Title: From behaviours to attributions: Further concerns regarding the evaluation of professionalism.
Authors: Ginsburg S, Regehr G, Mylopoulos M.
Reference: (2009)  Med Educ. 43(5):414-25. (PDF)

Title: Basing evaluation of professionalism on observable behaviours: a cautionary tale.
Authors: Ginsburg S, Regehr G, Lingard L. 
Reference: Acad Med 2004, 79:S1-S4.

Title: A sociological framing of medicine's modern-day professionalism movement.
Authors: Hafferty FW, Castellani B.
Reference: Med Educ. 2009;43(9):826-8. (PDF)

Title: Professionalism Perspective: Moving beyond nostalgia and motives: Towards a complexity science view of medical professionalism.
Authors: Hafferty FW & Levinson D.
Reference: (2008) Perspectives in Biology and Medicine, 51, 599-614. (PDF)

Title: The role of the student-teacher relationship in the formation of physicians: The hidden curriculum as process.
Authors: Haidet P & Stein HF.
Reference: (2006) Journal of General Internal Medicine 21, S16-S20. (PDF)

Title: Assessing how well three evaluation methods detect deficiencies in medical students' professionalism in two settings of an internal medicine clerkship.
Authors: Hemmer PA, Hawkins R, Jackson JL & Pangaro LN. (2000) 
Reference: Academic Medicine 75(2):167-173. (PDF)

Title: Proto-professionalism: how professionalisation occurs across the continuum of medical education
Authors: Hilton SR, Slotnick HB
Reference: (2005)  Medical Education, 39(1): 58-65. (PDF)

Title: A theoretical sketch of medical professionalism as a normative complex.
Authors: Holtman MC. 
Reference: Advances in Health Sciences Education (2008) 13, 233-245. (PDF)

Title: Perceptions of professionalism in medicine: A qualitative study.
Authors: Jha J, Bekker HL, Duffy SRG & Roberts TE.
Reference: (2006)  Medical Education 40(10):1027-1036. (PDF)

Title: Viewpoint: Professionalism in Modern Medicine: Does the Emperor Have Any Clothes?
Authors: Kinghorn WA, McEvoy MD, Michel A, Balboni M.
Reference: (2007) Academic Medicine, 82(1):40-45. (PDF)

Title: Medical school selection: screening for dysfunctional tendencies.
Authors: Knights JA & Kennedy BJ.
Reference: (2006) Med Educ 40(11):1058-64. (PDF)

Title: Assessing professionalism: a review of the literature.
Authors: Lynch DC, Surdyk PM, Eiser AR.
Reference: (2004) Medical Teacher 26(4):366-73. (PDF)

Title: Sociological interpretations of professionalism.
Authors: Martimianakis MA, Maniate J, Hodges BD.
Reference: (2009) Medical Education, Sept; 43(9):829-37. (PDF)

Title: Disciplinary action by medical boards and prior behavior in medical school.
Authors: Papadakis MA, Teharani A, Banach MA, Knettler R, Rattner SL, Stern DT, Veloski JJ, and Hodson CS.
Reference: (2005)  NEJM. 353(25): 2673-2682. (PDF)

Title: Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards.
Authors: Papadakis MA, Arnold GK, Blank LL, Holmboe ES, Lipner RS.
Reference: (2008) Ann Intern Med. 2008 Jun 3;148(11):869-76. (PDF)

Title: The "pyramid of professionalism": Seven years of experience with an integrated program of teaching, developing, and assessing professionalism among medical students.
Authors: Parker M, Litt M, Haida L, Zhang J, Wilkinsons D, Peterson R and Ozleins I. et al.
Reference: (2008) Academic Medicine 83(8): 733-741.  (PDF)

Title: Development of an instrument to measure the climate of professionalism in a clinical teaching environment.
Authors: Quaintance JL, Arnold L & Thompson GS.
Reference: (2008)  Academic Medicine 83, S5-8. (PDF)

Title: Viewpoint: The trouble with assessing students' professionalism: Theoretical insights from sociocognitive psychology.
Authors: Rees CE, Knight LV.
Reference: (2007)  Acad Med. 82(1):46-50. (PDF)

Title: Banning, detection, attribution and reaction: The role of assessors in constructing students' unprofessional behaviours.
Authors: Rees CE, Knight LV.
Reference: (2008) Med Educ. 42(2):125-7. (PDF)

Title: Professionalism in clerkships: an analysis of MSPE commentary.
Authors: Shea JA, O'Grady E, Wagner BR, Morris JB. & Morrison G.
Reference: (2008)  Academic Medicine 83, S1-4. (PDF)

Title: The prediction of professional behavior.
Authors: Stern DT, Frohna AZ & Gruppen LD.
Reference: (2005)  Medical Education 39(1):75-82. (PDF)

Title: The developing physician--becoming a professional.
Authors: Stern DT, Papadakis M.
Reference: (2006)  N Engl J Med 355(17):1794-9. (PDF)

Title: Professionalism Mini-Evaluation Exercise for medical residents in Japan: a pilot study.
Authors: Tsugawa Y, Tokuda Y, Ohbu S et al.
Reference:  Med Educ 2009: 43: 968-978. (PDF)

Title: Measuring professionalism: a review of studies with instruments reported in the literature between 1982 and 2002.
Authors: Veloski JJ, Fields SK, Boex JR, Blank LL.
Reference: (2005)  Acad Med 80(4):366-70. (PDF)

Title: Medical Professionalism: Crossing a generational divide.
Authors: Walsh C & Abelson HT.
Reference: (2008)  Perspectives in Biology and Medicine, 51, 554-565. (PDF)

Title: Cynicism about organizational change: Measurement, antecedents, and correlates.
Authors: Wanous, JP, Reichers AE & Austin JT.
Reference: (2000)  Group & Organization Management 25(2), 132-153. (PDF)

Title: A blueprint to assess professionalism: results of a systematic review.
Authors: Wilkinson T. J., Wade, W. B., and Knock L.D.
Reference: Acad Med. 2009; 84: 551-558. (PDF)

Books

Title: Measuring medical professionalism.
Authors: Stern D,
Reference: (2006)  New York : Oxford University Press

Articles relevant to the discussion of professionalism

Title: The hidden curriculum, ethics teaching, and the structure of medical-education
Authors: Hafferty FW, Franks R
Reference: (1994)  Academic Medicine 69 (11): 861-871. (PDF)

Title: Beyond curriculum reform: Confronting medicine's hidden curriculum
Authors: Hafferty FW
Reference: (1998) Academic Medicine, 73(4): 403-407. (PDF)

Title: Characterizing the Patient-Centeredness of Hidden Curricula in Medical Schools: Development and Validation of a New Measure
Authors: Haidet, P, Kelly PA, Chou C, et al.
Reference: (2005)  Academic Medicine, 80(1):44-50. (PDF)

Title: The devil is in the third year: A longitudinal study of erosion of empathy in medical school.
Authors: Hojat M, Vergare MJ, Maxwell K, Brainard G, Herrine SK, Isenberg GA, Veloski J and Gonnella JS.
Reference: (2009)  Academic Medicine 84, 1182-1191. (PDF)



Assessment for Selection

Title:
Assessing personal qualities in medical school admissions
Authors: M A Albanese, M H Snow, S E Skochelak, K N Huggett and P M Farrell
Reference: Acad Med 2003;78:313-21
Submitted by: David Prideaux
Date submitted: December 2009
 
Title:
Uncertainties in the selection of applicants for medical school
Authors: J Benbassat, R Baumal
eference: Adv Health Sci Educ Theory Pract 2007;12(4):509-21
Submitted by: David Prideaux
Date submitted: December 2009
 
Title:
Predictive validity of the multiple mini-interview for selecting medical trainees
Authors: K W Eva, H I Reiter, K Trinh, P Wasi, J Rosenfeld and G R Norman
Reference: Med Educ 2009;43:767-775
Submitted by: David Prideaux
Date submitted: December 2009
 
Title:
Graduate entry to medicine: widening academic and socio-demographic access
Authors: D James, E Ferguson, D Powis, I Symonds and J Yates
Reference: Med Educ 2008;42:294-300
Submitted by: David Prideaux
Date submitted: December 2009
 
Title:
Applicants to medical school: the value of predicted school leaving grades
Authors: A B Lumb, A Vail
Reference: Med Educ 1997;31:307-311
Submitted by: David Prideaux
Date submitted: December 2009

Title:
Personality scale validities increase throughout medical school
Authors: Lievens, Filip; Ones, Deniz S.; Dilchert, Stephan
Reference: Journal of Applied Psychology. Vol 94(6), Nov 2009, 1514-1535
Submitted by: Barry Issenberg
Date submitted: January 2010

Title: Medical student selection in New Zealand: looking to the future
Authors: PJ Poole, HJ Moriarty, AM Wearn, TJ Wilkinson, and JM Weller
Reference: N Z Med J 1 Jan 2009 122(1306): p. 88.
Submitted by: David Powis
Date submitted: March 2010

Title: Improving the selection of medical students
Author: David Powis
Reference: BMJ 2010;340 c708
Submitted by: David Powis
Date submitted: March 2010