Talent ID 2.0: Stop Measuring Performance, and Start Testing Temperament

»

skill-scanDo you have an eye for identifying talent?

Can you watch people perform, talk to them, and then choose the person who’s destined to succeed in the long run?

Most of us instinctively answer “yes,” because it feels like we do.

In fact, science shows us that we’re mostly flattering ourselves. Because the truth is, long-term success is extraordinarily difficult to predict. Interviews are notoriously unreliable. Sports drafts, in particular, are expensive casinos.

The problem is that a person’s progress ultimately depends on factors that are extraordinarily difficult to measure — stuff like character, emotions, discipline, motivation. How do they respond to failure? What’s their vision for themselves? Can they persevere in the toughest situations?

We call this “the soft stuff” but in fact it’s not soft at all — it’s the hardest, most vital stuff there is.

The real question is, how do you measure it?

I came across a great answer developed by San Francisco 49ers coach Jim Harbaugh. Harbaugh, a former NFL quarterback before becoming a successful coach, has developed a simple way to measure the soft stuff of his quarterback and receiver prospects.

He plays catch with them.

That’s right — he plays catch, throwing a football back and forth. He does this at pro days, when prospective draftees try out for an audience of coaches and scouts. Every other NFL coach treats the event as a spectator sport, standing on the sidelines with clipboards and video cameras.  Harbaugh, on the other hand, uses it as an opportunity to engage.

Here’s the trick: with Harbaugh, it’s not an ordinary game of catch. Because after a few warmups, Harbaugh starts throwing harder, with more and more intensity. He makes the player run out for passes, making tough throws. He challenges the player, sees if they instinctively rise to the occasion. Some players back down, get uncomfortable. Others embrace it. From the Wall Street Journal:

Harbaugh first took a liking to [Colin] Kaepernick, who played in college at Nevada, when they played a supercharged game of catch at his pro day in Reno. Harbaugh threw hard; Kaepernick threw harder. Kaepernick, Harbaugh came to understand, had the drive he was looking for. Although he wasn’t considered a top prospect—San Francisco took him in the second round in 2011—Kaepernick has started in two straight NFC Championship games and led the 49ers to the Super Bowl in the 2012 season.

I love Harbaugh’s litmus test because it measures two things at once: interpersonal chemistry and competitiveness. It operates at the gut level, where the most important factors reside.

In short, this is not talent ID — it’s temperament ID.

It reminds me of a master teacher I researched at the Bolshoi Ballet, who tested new students by teaching them a difficult and strange new move that none of them had ever done before. The teacher wasn’t interested in how well they performed so much as whether they embraced the process. Did they rise to the challenge? Did they struggle well? Like Harbaugh’s test, it was a gut-level litmus test of temperament and character.

The next question: are there ways to apply this idea to other disciplines? What’s the business version of Harbaugh method? What’s the music version?

Do you know of any similar temperament-ID tests that might be worth sharing?


Rate This

1 Star2 Stars3 Stars4 Stars5 Stars (5 votes, average: 4.40 out of 5)
Loading ... Loading ...

Share This

Bookmark and Share

21 Responses to “Talent ID 2.0: Stop Measuring Performance, and Start Testing Temperament”

  1. Hap says:

    A therapist I knew interviewed prospective admin assistants by asking them to drive him somewhere. The interior condition of their car told one story, and their driving style another.

  2. doc says:

    Sounds like Harbaugh adapted Carol Dwecks studies to football.

  3. djcoyle says:

    Thanks, Hap — that is fantastic.

    And for some reason it reminds me of a successful college basketball coach who had a rule: if the prospect made any excuse during their conversation (about being late, or why his GPA wasn’t higher, or why the house wasn’t clean — literally anything at all — that coach would not sign that player. The thinking is the same as with your therapist example: how you do anything is how you do everything.

    The deeper question: do micro-samples taken from personal life matter? I must confess, there are times in my life where my car isn’t exactly the cleanest — should that mean that I shouldn’t be hired?

  4. Scott says:

    That’s the problem with micro-samples – you may miss the diamond in the rough, but having to make snap decisions with imperfect information it will generally be right (or good enough).

    One thing I look for in young tennis players is who has poor temperament in terms of crying/whining/racket throwing – these are not necessarily bad (if not about playing the game). If looked at another way than poor sportsmanship, they indicate passion or hating to lose. For example, Djokovic, Federer, Borg all had terrible tempers, which they later mastered. But at an early age their passion to win or play well was evident in how they hated to lose or play poorly. I want a young player who has passion over an indifferent player – the temper can be tamed, but the you can’t instill the fire to win.

    See this video of young Djokovic – nothing terribly impressive about his tennis, but he sure whined a lot. That’s ok!

    https://www.youtube.com/watch?v=Oeid5ioDF2Y

  5. djcoyle says:

    Hey Scott, Great points — thanks so much. Emotional skills develop later. But the obsession/love for winning are often there from the start, and they tend to create some turbulence. And great video of Djokovic! His whining is world-class, for sure.

  6. John says:

    Hi Daniel,

    Great Post!

    In my opinion, the business version of this test is how someone responds to seeing there company go bankrupt. Do they get back up or they simply quit.

  7. John says:

    Daniel,

    I just wanted to comment on your post

    “The thinking is the same as with your therapist example: how you do anything is how you do everything.”

    Another great quote I have heard in the same vein is…

    “How you do the small things is how you do the big things” – Brendon Burchard

  8. Michael Binder says:

    Great post. I have red your book already about 5 times and I am following with great interest your blog. About this article I am still thinking about how to use it in music. I am a doublebass teacher in Spain, so I am very interested about topics like talent, practise and performance. I think there is no talent, not in the meaning we normally define talent. Students who have more difficulties at the beginning, do not need to be less sucessfull later. It is true that we are speaking about deditation, commitent and passion. Maybe as well we are speaking about suffering. Practise is not always great fun, cool and hip. It is hard, boring and sometimes you really do not why are you doing it, but later on stage you see it is worth. There are no shortcuts! Really great performers are really thinking only in what they are doing. Thats their first, second and third thought, later comes the rest of the normal life! It is a question of if you are really commited to do everything, really everthing for being the greatest or not. And it does not matter if it is in sports, music, art, business, family life etc.
    Maybe it is possible to have other opinions from the readers of your blog.
    Thank you and greetings from Spain,
    Michael Binder

  9. djcoyle says:

    Hey John, That is a tremendous quote — thanks for sharing it. It reminds me of a story I heard from a chef friend of mine about a famous culinary teacher who gives prospective students just one test: to prepare a single scrambled egg. That’s it. But he watches how they do it — the care they take, the details they attend to — and that tells the teacher all he needs to know about the student. Little = big.

  10. Alex says:

    I am extremely organized with some things – notes, training, tablet, books, study… – and sloppy with others – my car and it’s interior for an example. My driving style varies with my goal, I like to be smooth (fast or slow, aggressive or defensive).

    It probably is a decent test when hiring a secretary.

    I do look for is the same thing the coach does, how they respond to challenge, do they look for a solution or for an excuse and someone to blame. Do they trust in themselves to rise up to the occasion or do they distance themselves emotionally to preserve their ego (if you try really hard and fail then you can’t rationalize to yourself that you just didn’t wanted).

  11. Alex says:

    Also, these things can be worked on, growing in a healthy environment helps a lot.

  12. Chris says:

    Coaching soccer at the DIII college level I put juggling standards in place for each class (each year you are expected to improve and meet the next goal). Juggling is difficult to do because it has a high failure rate and it requires a lot of deep practice. Your score does not necessarily determine if you play but it does give me a great indication of your commitment – what you do when no one else is watching.

  13. Jim T says:

    I would have the basketball team I coached meet standards in a mile run each Fall before the official practice started. Running the mile has nothing to do with abilities to play basketball and times didn’t mean as much as their attitude. Demonstrating they could compete in that 3rd and 4th lap, and more importantly showing how they approached the preparation to get better was what I looked for. The process of getting in shape to be able to practice at an intensity that would allow them to get better is a hard concept to get across.

  14. John says:

    As a parent of two, I am curious if this can be taught. My first born is very timid. We even have a “try new things” chart on our fridge hoping to encourage him to, well, try new things. We’ve had it up for a while and there is little progress, even with milestone incentives. Is this just who he is in his core? How can I draw out of him a desire to keep trying and try new things?

  15. Kevin says:

    John,
    I see similar situations over and over as a youth soccer coach with the first born having less of a competitive drive. Sibling rivalry tends to make the younger kids more competitive. Are there any parents out there with a first born who is ultra competitive???

  16. doc says:

    John & Kevin, Openculture.com has 950 free courses of all disciplines. One of the courses, Science for the Non Science Major by Jay Phelan, has a lecture or 2 on risk taking. He talks about an evolutionary and physiological reason why some people are bigger risk takers than others. However, he does not believe you are locked into that behavior one way or the other. Follows along the nature/nurture thinking. Unfortunately, I’m not sure you can tell from the website which exact lecture it is until you begin watching (no table of content that I could find). He is an excellent instructor and when he finished his last lecture (there are 19)the students gave him a standing ovation. It was the best course I have ever watched or attended. He has also written a book called “Mean Genes” which was the text for the course. It is also in kindle format for about $10. I would recommend this course for anyone wanting to know more about the science of life without having to get into the technicalities of a science major course.

  17. Kurt says:

    I remember reading a story about Bill Belichick and his approach to interviewing draft prospects prior to the draft. He would take them individually into a video room and show them a play from the past season in which the prospect made a mistake. Belichick then observed how the prospect reacted – did he blame his teammates or hold himself accountable?

  18. bill dorenkott says:

    Current National Team Director of USA Swimming, Frank Busch, would play hoops or ping pong with recruits when he was the head coach at Arizona. My guess, is he was simply looking for kids who hated to lose.

  19. Victoria says:

    In response to your question about tests for other disciplines: Playing Pictionary might be a good one for an artist. An oral storytelling game, where each person adds a couple of sentences, might be interesting for someone looking for a person to do any sort of writing. For music, a party game I once saw a group of my Bluegrass musician friends play, would probably qualify. Three containers: one with all the instruments that were available, each written on separate slip of paper; another with each musician’s name on a slip of paper;; and the third with the names of Bluegrass songs, and tunes, called out by the group. Five names were then pulled, and each person drew an instrument (they had to draw again if they got their own best instrument). Someone in the group at large then drew a song, and the chosen musicians had to try to play it. This could be played with other genres of music; but wouldn’t work for any wind instruments. Also for music, name a genre and have the person improvise. Actors also do improv; but the whole audition process is pretty much a test of this sort.

  20. Erik says:

    As an engineering manager I used the situational interview. My father taught me that people are hired for what they know (resume) and fired for who they are (performance). So during our situational interviews we would have a few questions that put the person in impossible situations with competing concerns. Every time they “found a way out” we changed the game. Some quit, some rose to the challenge, keep finding new ways to attack the problem and subsequently were hired.

  21. Some years ago I was discussing ‘talent identification’ of swimmers with a Polish weight lifting coach. I asked what test he gave aspiring weight lifters. ‘Chin ups,’ came the reply. I asked how many chins he was expecting from outstanding athletes? ‘Don’t care,’ he said, ‘I want to watch how they fail.”

Comment On This