Monthly Archives: April 2012

Thinking About Admission Biases

Cracking the admissions process (Photo: grandmaitre)

I have been recently reading Thinking Fast and Slow by Daniel Kahnmen, the only psychologist to have ever won the Nobel prize in economics for his work in decision making and behavioral economics. Based on over three decades work on cognitive psychology, Thinking Fast and Slow delves into how our brain thinks, comparing our subconscious intuition and our deliberate cognition.

Apart from being a fantastic read, probably the best book I have read in, it got me thinking about how we select medical students. Long time readers may know that I help out at my medical school’s admission committee with choosing applicants each year. Our school is not unique in its way of selecting from the applicant pool. We use a criteria of GPA, MCAT, Interview Scores, Extracurricular Activities and References to score and rank each applicant.

After reading this book, I am more aware of many flaws in the current system. Specifically, when it comes to the many methods and heuristics that we use in evaluating applicants. Heuristics being problem-solving methods our brain uses to solve complex problems by experience. a mental short-cut if you will.

Evaluating Attributes

One of the cognitive biases that medical school admissions use is attribute substitution. This heuristic occurs when an individual has to make a judgment of a target attribute that is complex. Instead of having to work through the more complicated problem, you substitute a more easily calculated attribute.

Rather than having to evaluate students holistically, medical schools use quick substitutions.  GPA are used as a measure of diligence, the MCAT of raw intelligence and interview scores a test of interpersonal communications. There is little to no evidence that these criteria will select great doctors, but hundreds of medical schools continue to use these criteria. For example, when an applicant who has good grades and a high MCAT score but did poorly on their interviews, they are automatically judged as someone who is a bookworm, even when they make in fact be outgoing and social. More important attributes such as empathy, communications and integrity are left ignored as they are difficult to assess.

Anchoring Biases

Similarly, anchoring is a commonly seen phenomenon. It is a cognitive bias that makes one piece of information weigh more heavily than it should when it comes to decision making. I previously wrote in a post that the best way to increase your chances of medical school is to Not Get Eliminated. I didn’t truly understand at the time why this was true other than by personal experience working on the adcom. After reading Thinking Fast and Slow I can see that the admission committee often gets blinded by anchoring biases.

When the admission committee reviews which applicant to accept, they are more drawn to red flags that then effectively “blinds” them to neglect the rest of the application. If an applicant said an off-hand remark during the interview, they are all of a sudden seen as unprofessional and deemed unfit for a career in medicine. Similarly, a negative sentence in a reference letter might just sink your whole application.  It becomes anchored in the minds of the admission committee and becomes weighted more negatively than it should. Even if the rest of the application is outstanding, “red flags” can create an anchor bias and lead us to make insufficient judgments.

An Imperfect Process

After reading Thinking Fast and Slow and understanding all the different heuristic methods our brains employ, it’s not hard to see the flaws in the current admissions process.

The first point is that it is extremely difficult to predict whether an applicant will be a good physician or not. We are given scant information, some academic numbers and a quick interview before we have to decide whether to accept them or not. There’s not enough concrete evidence to know whether someone will be an innovative leader or whether they will be involved in multiple lawsuits.

Secondly, we employ many cognitive biases that ends up screening out great applicants. Someone who is quieter and shyer may in fact be a great physician one day, but we might score them poorly on the interview because they don’t have as much to say. Every admission committee member has their own biases as to what attributes to look for. Even the discrepancy between different adcom members can affect the application process. There is a lot of variability on how well your application would score based just who reviews your file.

Finally, there’s not a lot of data for the correlation between the criteria to select a medical student to how they will eventually perform as a doctor. We have to realize that we need to have all sorts of doctors from including surgeons, family doctors and even pathologists. And I think it’s naive to believe we know how to best select those will succeed in each field. Especially if all we are trying to do is extrapolate marks and extracurricular activities.

Where to go from here?

The current admissions process is flawed but by no way is it broken. Medical schools still manage to graduate great doctors every year and have no problems filling their classes with amazing applicants.

But in order to improve, I think admission committees have to be more aware of the potential heuristics and biases that we have when we evaluate medical school applicants. Furthermore, there needs to be a stronger push towards a more holistic evaluation of individuals instead of one where each person is evaluated on their numerical scores. And above all we need to make the admissions process as fair and transparent as possible, so that great applicants are not excluded for the wrong reasons and that bad applicants do not end up gaming the system.

If you want a sample of what Thinking Fast and Slow is like, the Nobel prize site has an excellent lecture that summarizes all the main points of the book. I also didn’t realize the influence of Kahneman’s work. It was the basis of many of the books I have read in including How Doctors Think, Blink, Paradox of Choice

 

Learning Smarter and Better in Medical School

To study the phenomenon of disease without books is to sail an uncharted sea, while to study books without patients is not to go to sea at all – William Osler

Medical School – One Last Exam

Fourth year is flying by fast and I’m in the final stretch before my licensing exams. It’s pretty neat to see how much medical knowledge you learn (and forget!) in just four years. While revisiting old notes, I’m surprised as to how much deeper I now understood and appreciate topics after some clinical experience. For example, long term management of diabetes can be quite boring in your first year but after you’ve seen many patients with it, learning becomes more meaningful. Even the weird and wonderful zebras become more fun to read about.

There is however one big regret I have about my learning during medical school.

I wish I had organized my “knowledge” better

Classroom Learning – A Flawed Approach

In the first two years of medical school, you spend a lot of time in the classroom. You listen to a lecturer speak, you go through the ppt slides and you try to summarize all the facts in a way you can understand. You cram all this knowledge in your head and regurgitate it for an exam.

This method works pretty well for passing your courses. It’s not a great approach for mastering clinical knowledge.

Classroom learning can be dangerous for several reasons. One is that your learning becomes centered around what teachers present to you, instead of on what you should and need to know. It’s easy to be complacent and have knowledge spoon fed for you. You start to memorize minutiae and trivia questions because that is often what they put on the test. You stop thinking and asking questions beyond the scope of the course, and instead focus on maximizing your grades. The classroom teaching often narrows the scope of your knowledge.

And the worst part is that the school system encourages this type of learning. You do well for being able to recite all the fine details that was taught in class. You get marks for remembering minutiae. You pass the exam, your marks are good, you feel reassured.

Learning Outside the Classroom

It’s not until your clinical years when you realize that learning from what they only give you can be a blunder. What you have learned in the classroom might not be relevant knowledge. On the wards, there is no longer someone telling you what you need to know to do your job well. You have to determine what knowledge you need to acquire to be a successful medical student.

You might still refer to your class notes initially but you soon begin searching for answers elsewhere. You start to learn from many different resources. When there was a topic I wanted to know more about I would consort more comprehensive textbooks, read journal articles and electronic references such as UpToDate.  I picked up clinical pearls from nurses, residents and staff doctors. I learned through experience and identified gaps in my knowledge that I could address.

My Dilemma – The Consolidation of Knowledge

The problem I face currently is consolidating all this knowledge. It can be quite tricky to put everything you learned into a system that works for you – one which you can translate into clinical knowledge. What I found in my clinical  years was that it was easy to study for each rotation by itself. You choose the right resources and you can focus your attention. The hard part is when you try to consolidate all these separate pieces of information together.

My Previous Solution: During third year, I separated each section of Toronto Notes as a basic study aid during my rotations. I would read around my cases and annotate any additional information in the margins of my notes. This turned out to be a sufficient method in creating a comprehensive guide to each specialty. I read every section of TO notes while marking it up with my own mnemonics and flow charts. I used a different approach to each rotation and it worked pretty well.

The Current Problem: What I face now is creating a consistent learning system that can capture information and turn it into knowledge. I have become adept at passing individual exams but I want to improve my learning to a point where I can be a master clinician.  Going into residency, I will have to start understanding each topic more in depth and be more evidence-based with my knowledge. It will require newer and better ways of learning.

Another issue is I am at a crossroad for is deciding on what medium to learn from. I enjoy studying from paper notes but find it lacks the flexibility of digital notes. I remember I had tried implementing Dr. Brandt’s strategy of index cards and her studying system but found that it wasn’t possible to keep up with the paperwork involved. I have tried digital strategies like OneNote but have found creating notes from scratch a time-consuming process.

An Ideal System

In a perfect world, I would remember everything I read, file it away in the right folder in my brain and have the ability to recall it whenever I wanted.

For those less than super-human, creating a study system is crucial to long-term learning. As I continue my medical training, my learning continues to evolve. What worked for me in the past might not be the best solution for my current studies.

My Goals, Strategies and Tools to Get There

My objectives are to have a deep and long term understanding of medical knowledge that I can apply clinically. I want to further develop a system of life-long learning and skills development. Passing my exams will be a measurement of my progress and not the end in itself.

Strategies I plan to use for residency include

  • Reading around patient cases and related topics
  • Aiming to learn something everyday – even if it’s on a topic I have gone over a dozen times
  • Be inquisitive and come up with relevant clinical questions and find the answers to it
  • Creating concise key summary notes that I can refer back to and update

Being more Evidence Based – Some tools I have started using to keep track of medical literature include Mendeley for keeping track of journal articles and Xmarks for organizing webpages. For keeping up to date with new medical literature I subscribe to the CMA’s Infopoems and the ACP Journalwise. I also subscribe to NEJM’s clinical cases and images via RSS.

Doing my Homework, Reading Textbooks – Next year, I’ll eventually purchase one of  “big texts” for internal medicine (deciding between Harrison’s or Cecil’s) and break it up into readable chunks. If I am motivated and lucky, I hope to get through it before my PGY3 year.

Notes, Notes, Notes – Finally, I have yet to determine if my notes will be paper, digital or some type of hybrid.

Experimenting – Just like how medical school was a new challenge, I’m sure residency will have its unique challenges. Being open and adaptable will be an important part on this new journey.

I guess you don’t really think about consolidating and organizing everything you have learned in medical school until a big exam like this forces you to do it. Just as we went from elementary school to high school, university to medical school, residency will require a new higher level of learning.

Would love to hear your opinion on what kind of a learning system you use? How do you collect your information and process it into knowledge? Are there any tools that you use to make learning more effective and efficient?

 

Medical Student Personalities and Performance

You may have seen this comic strip on the 12 types of Med students by Michelle Au. There are all sorts of personalities in medicine and they often correlate with a medical specialty too!

Research on Medical Student Personalities

A recent research article titled Associations of Medical Student Personality and Health/Wellness Characteristics With Their Medical School Performance Across the Curriculum got me thinking about personality types and performance in medical school.

The authors did a retrospective study of medical students who had just finished their first clinical year (3rd year). What the results show was that personality traits of conscientiousness, extraversion, and empathy were strong predictors of clinical skills, interpersonal behavior and humanism.

Looking through the literature, there was another study a couple years back that looked at Medical students’ personality characteristics and academic performance: a five-factor model perspective. They similarly concluded that extraversion and agreeableness were important in interpersonal communication in a clinical setting.

Self Reflection on My Own Personality and Performance

These studies and having recently finished reading Success on the Wards got me thinking about my own personality type and its influence on my medical school performance.

Out of the Big Five Personality Traits, I would probably score high in Openness, Conscientiousness, Agreeableness, low in Neuroticism (the lower the better) and Extraversion.  Not surprising, my clinical rotation evaluations often reflected these traits. Many preceptors have commented on my excellent organization, preparedness and empathy. However, many have also encouraged me to participate more in discussions, a shortcoming of my introversion.

How Can Identifying Personality Types Make Better Doctors?

I think identifying and measuring a student’s personality traits can be a powerful tool.

For myself, I know my introversion may be a source of miscommunication. With this knowledge I can make a concerted effort to talk more with other team members despite my introversion. Similarly, you can find role models who have traits you lack and emulate their behaviors.

Another way that personality types can help shape our future doctors is in through our medical school admissions. Currently schools mainly use a number based system (GPA, MCAT) in making their decisions. But clearly the numbers don’t tell the whole story.

In a way, we are using these numerical criteria as an indirect measure of personality traits. Applicants who have high grades tend to exhibit conscientiousness. Applicants who volunteer are probably compassionate and empathetic. Applicants who have leadership positions probably have extraversion and confidence.  If there were a way to measure an applicant’s personality traits directly and objectively, it might even be a better metric than what we use now.

What Makes a Good Doctor?

Ultimately, medical schools want to produce good doctors who will show empathetic care to their patients, inquisitiveness in their research and leadership in their health care teams. To think that looking at a GPA, MCAT score and list of extracurriculars can determine what kind of a doctor an applicant will be is a joke. Unfortunately, it’s very hard to predict what makes a good doctor. I can only hope admissions continue to evolve to a holistic evaluation of an individual.

The new MCAT 2015 is already hoping to address some of these deficits by adding a social and behavioral sciences section. Perhaps there will be a personality assessment in the near future. Who knows what will come next?

Do You Think Personality Types Matter in Medical School Admissions and our Doctors?

 

 

Med School Admission Strategies – Don’t Get Eliminated

What can you do to maximize your medical school chances? (Louish)

Each year, I help review the hundreds of applications we receive at our medical school. Each year, I’m impressed by so many great applicants. It seems like each class just keeps getting better.  So in a pool of hundreds of applicants, many who I have no doubt would make  fine doctors, how do we differentiate each applicant and select a medical class?

Stand out, by NOT standing OUT in the wrong way

Conventional advice is that you should do well in everything. A perfect applicant is someone with a 4.0 GPA, 42T MCAT, tons of research, leadership positions, volunteering opportunities, stellar references and an outstanding interview. On top of all that, you must excel in one thing that makes you unique and memorable.

In reality, I don’t think that’s a realistic goal for most applicants. In fact, only a handful of those “perfect” candidates exist. Instead medical classes are made up of a more diverse group of people. Some medical students have taken time off before applying while others have had alternate careers. Many people have grades that are great but not perfect and MCAT scores that are good enough. They may be play the piano but are not anywhere near a concert pianist.

I think a much better strategy for getting into medical school is to NOT stand out in the wrong way.

You see, when every application is superb and similar, it’s hard to deem one as better than another. It is much easier instead to look for red flags that can help us eliminate an applicant. Finding the right people for medical school is not so much a process of choosing the right applicants as it is a process of elimination.

Your Application is Only as Strong as the Weakest Link

I often get emails from people asking which extracurricular they can do to stand out in the eyes of the adcom. What unique leadership position should they be in to get a better chance for medical school? Would an exceptional research project overcome a poor freshman GPA?

Time and time again, I tell them the same message. Grades matter first, and then your MCAT and finally your extracurriculars / references. If there’s something that’s going to eliminate you first, it’s your GPA and MCAT scores. Without meeting these requirements, the rest of your application doesn’t even get looked at!

Similarly, when we interview students, it’s far easier to spot applicants with red flags than it is to differentiate great applicants. If they seem unethical, unable to speak or have questionable responses, they will be cut. It’s the same with reference letters. Most letters will read the same. When you’re reading through hundreds of them, very few letters will leave a lasting impression. The admission committee is instead trying to screen out students that might be a problem in the future.

It’s about Picking Out the Rotten Apples

When there are twice as many great applicants as there are spots, as a medical school your main concern is to make sure you weed out the bad apples.

Because who’s to say applicant #140 is that much of a better student than #220 who just missed a spot. I’m sure they both could be good doctors. The supply of great applicants far exceeds the number of medical school seats.

Instead, it’s much more important to weed out the psychopaths and sociopaths who will be troublesome down the road. Medical schools aren’t worried about not filling their class with great people, they are worried about letting in people that will cause them headaches in the future.

Don’t Focus on being Perfect, Minimize Your Weaknesses Instead

A more realistic strategy for medical school admissions is to make sure your application does not have a fatal flaw. A bad grade could be what’s keeping you from getting an interview. An unbalanced MCAT score with one low section could be the difference.

Similarly, writing something controversial in your essays or choosing the wrong referee may sink your application.

So stop trying to be perfect with your medical school admissions. Realize that medical schools are looking more for people without any major flaws than the perfect applicant. A person with perfect grades, lots of extracurricular activities and awesome interview skills is still a bad applicant if they have something that doesn’t seem right, especially if they are unethical or lying.

Evaluate your application, is there something that’s holding you back? Let me know in the comments or by email, and I will give you my feedback!