Saturday, February 19, 2011

How I read a Truman file

I finished my Udall blog with an extended post on "How I read a Udall file." I thought I would do the same with Truman -- take the application question by question, piece by piece to show what I was looking for as I read.

First, the big caveat: I am one reader. Other readers read differently. It's a subjective process. Readers often disagree on files (today we had one file that all three of my team members needed to read -- it was scored as a 7.5, 7.0, and 4.0. The two of us that had scored it high had to do some serious convincing of the third. She had some good points I hadn't thought of, but the other reader and I were really convinced by this file, and she ended up as a finalist). So take what I have to say with a large grain of salt - the other 17 readers would each differ with what I write here, I am sure.

FacRep letter: I already wrote about this yesterday, but this is where you first encounter the candidate. I am looking to learn the theme of the application, get a preview of what the highlights of the application will be, get any contextual or background info that is really needed (family troubles, medical issues, etc that might not show up elsewhere), and get a sense of how much the FacRep really thinks this one deserves a Truman. I also look for what's missing. Three long paragraphs about academics, and no mention of leadership? Uh-oh. FacRep letters really need to make the case in all three areas -- appropriateness of grad plans, leadership, and service. Personality stuff -- how much you like talking with him or look forward to your meetings with her -- usually don't really matter.

I then glance through the opening materials -- what is their undergrad degree in? What do they want to go to grad school in, and for what degree? This starts to frame what I am about to read -- the candidate's background. Knowing what the end goal is tells me what we are aiming for in reading their activities of the last few years.

The Washington Summer Institute is actually a quite revealing question and helps to let me know a surprising amount about the student. How enthusiastic are they about this opportunity? How much have they thought about and researched opportunities? Are they creative? What do they care about, and why? Are they willing to push their comfort zone and try something different? Can they tell me all of this in a very short amount of space? Again, this helps to give me a preview of where the application is going to end up, so I have some idea of what roads we might take to get there. And, no, they do NOT have to say yes to this question -- but if they say no, they should say why that is, and should have a really good reason for it (Teach For America starts training in the summer, for example). Since so many readers are past scholars who know that the Truman is so, so much more than money for grad school, they want to see that candidates know the true value of the award lies in opportunities such as TSLW and the summer institute.

Questions 1-6 -- the biographical information -- is really important. How students list their information (since it is in order of importance), where they have spent their time, whether or not there appears to be leadership activities, if what the FacRep highlighted seems to be showing up here, how long they spent on various activities, whether or not I can spot the emerging theme and evidence that they have led and done service work... it's all here, in a shockingly small number of words. If candidates have done something that involves an acronym or a campus-only group, they should briefly (briefly!) explain it, especially if they are not going to write about it in questions 7 or 8. They don't have to have something for every question, and very few candidates have something to list of substance in every box, but there should be something really substantive in question 2, something that demonstrates service (preferably hands-on) in question 3, and you've got to give us something to hang our hat on in question 4. I didn't look too thoroughly at questions 5 & 6. Really, I was mostly looking at two and three, and if there wasn't much there or in four, the candidate was already in trouble. When I finished these, I could usually guess what the leadership essay and public service essay were going to be about -- and if I had no idea, then that usually didn't bode well for those essays.

Truman has phenomenal resources on their website on how to approach the short-answer questions -- the advice you will get there is better than anything I can offer, really. I can tell you that question 7 is, to me, the most important part of the application. We're looking for leaders and change agents. We're looking for people who have already made a difference. "Lead by example" doesn't get you far. I found the sports leadership essays to not be very effective, with specific exceptions where the candidate really did show leadership that could be more broadly construed. And personally, I was much more interested in the essays that just stuck to the question and told me what they did, rather than the ones that turned it into a discourse on their personal views on leadership. If you don't have me on-board with your application by question seven, you're probably not getting me on-board. I should know who you are, how you spent your time, and what difference you have already made in the world.

Question 8 is on public service. This is where you show me that you care, and that you can't imagine a career not in public service. It's also a chance to highlight another major accomplishment, but it's not a place to just recite your resume -- it's a place to let the reader in and show that you care and what your motivation is. I need to know why you do what you do, so that I know that you won't just grab the money in the private sector later in life. 7 and 8 give you the chance to discuss just two things you have done in-depth, so you need to make really good use of them. A good question 8 answer (and there were surprisingly few of them) helps build a crescendo to..

Question 9! The turn! Now we know who you are, what you've done, how you prioritize and spend your time, the change you have created, and why you care. That's a lot. And the past is prologue to....oh, please let it make sense. There were so many candidates where I read question 9 and thought "What?! Where on earth did that come from?" If I thought that, the file almost never advanced. What comes next should be a logical and natural extension of the past. I did not read 9 terribly closely -- I looked to see if the candidate made sense of an issue, defined it well, and could speak knowledgeably about it. But both it and 10 (which I just glanced at) I take as being more fodder for interviews than a decision-maker on whether or not to advance as a finalist. I'll put the policy proposal in that category, too. I looked at policy proposals (when I got to them in the file) but did not look at them very closely, and can only remember one or two cases where it changed my mind on a candidate (not always positively).

I really liked reading questions 11-13, which I read as a group. They should be consistent with the past, of course, and definitely in line with what the candidate wrote about in question 9. But you learn a lot about the nominees by reading these -- creativity, vision, how they conceive of themselves, what they really want to work on, how much effort they put into the application, etc. I was surprised to find that 12 and 13 were more elucidating than was 11. Pretty much everyone wants to go to the same grad programs (and it is helpful when someone says "I want to go to ______ law school," and the dean of admissions at that law school is seated next to me). But after grad school? There are SO many paths their lives can take after grad school, and it is really exciting to see what they have come up with.

At that point, I have a pretty darn good idea of whether or not this kid is a finalist. The ones on the bubble can be pushed one way or the other by the letters of rec, but they're pretty uniformly positive. One piece of advice we can all share with our letter-writers, though: the "she is the pinnacle/perfect/ultimate/best/ideal Truman candidate" label is not a good idea. Because if she's not, that makes me wonder how well you understand this opportunity and how seriously I should take the rest of your letter. The best file I read all weekend -- a truly remarkable person -- had these beautifully understated letters. They didn't need to oversell the candidate, because they knew that the file would take care of itself.

I hope that helps. I have to say that much of what I have just written does not feel like it will be news to FacReps who have been doing this for some time, who go to conference presentations, who use all of the many pieces of advice on the Truman website, who reach out to Tara with questions, and who consistently read the NAFA listserv. Truman has a real commitment to transparency that I applaud and think makes for stronger applications from all candidates. It's near-impossible for a student to fake a strong question 7, for example, even if they know what they are supposed to write. Even if you know how to make a hamburger, if the beef is lousy, the burger will be, too.

Which is a roundabout way to address the questions that came up on the NAFA listserv about my serving on this committee as a FacRep, and whether or not my candidates will get an unfair advantage in the future by my reading several dozen files. Did I learn from this? Definitely. But most of what I learned was a reinforcement of past lessons learned in other ways -- there was not a lot about reading files that surprised me. It was quite interesting, and I am very glad I did it, and I think I may have a better guess in the future about whether or not my candidates will advance in the competition -- but a lot of that also depends on the region they are in, how competitive the state is, and issues of subjectivity and luck that none of us can control, no matter how many of these files we read. I do think my candidates will benefit from me being here -- but I hope yours will, too, from this blog.

Congratulations to all of the finalists! Go knock 'em out at the interview, and don't forget to enjoy getting to know one another -- it's fun to think of all of these files coming to life and sitting together in a room, sharing ideas and experiences.

To the non-selected candidates, remember that this is not a referendum on you as a person. I read many, many files of amazing young people who care, who lead, who are making a difference, and who are not going to be Trumans. You're all going to go on to lead successful lives, lives of service, lives of leadership. You won't do it with a Truman. That's ok. Yes, today stinks, and you should let it stink. But then get back to work, and go make a difference.

That's it from Annapolis, but I'm happy to answer any questions you have.

I'll see you in appeals!

My reading team is done, having started and finished our last region this morning. Just two states, but 31 applications, from which we selected 11 as finalists. One state was much more competitive than the other -- we took 3/13 from one, but 8/18 in the other. There were a few excellent applications, including one which we considered giving the label of "outstanding" to. The foundation asks us to reserve that designation for truly deserving files, and this one was awfully close. One of the three of us was convinced -- the other two (including me) thought it was close, but not quite in that category. We did not name any of the 104 files we read in the last 48 hours as "outstanding."

Now, of course, my thoughts are on the same topic as many FacReps' -- the appeals process. One of my four nominees has been named a finalist; the other three candidates from Grinnell did not advance. I had a good conversation with Tara about generalities of the appeals process -- not "which of mine should I appeal?" but "how does this work, and, in general, how should FacReps decide which non-selected candidate to put forward?" Here are her thoughts:

1) They are really looking for candidates who were overlooked in the selection process. These are often non-traditional candidates, transfer students, and others whose file may present some challenges to read.
2) Each region has about 2-3 spots to fill, maximum, from the appeals process. States don't matter at this point -- they are all considered together in one region. Some regions do have more candidates than others, though.
3) Appeals are all read on Friday by a team of about three people -- Tara, the most senior reader (20+ years of experience), and a third reader from the greater DC area. None of them re-read files that they scored earlier in the week. They do not take the original readers' notes and scores into account.
4) I asked if there is any strategy in appealing every year (as I do) vs. only appealing when the FacRep feels that there has been a major oversight (as some others do). She did not have a solid recommendation on this -- they do not know all of the FacReps well enough to know "this person never appeals, so take it seriously" or "this person appeals every year, so just ignore him." An appeal is an appeal, and they don't look at the institution when they look at the candidate.
5) Really, all she needs is the name and state. REALLY. Don't take time to send her more than that -- your finely-crafted paragraph explaining why the candidate's non-selection was a mistake won't impact the appeal process at all.

Bottom line: which one of your non-selected candidates has the best record of service, leadership, and a solid academic plan for the future? Appeal that one.

I also wanted to add a note about academics in the Truman competition. One of my big realizations this year is that this part of the Truman score is not about academic ability - it is about the appropriateness of the graduate school plan for this student, given their academic record and plans for the future. A 4.0 who plans to go to the Kennedy School is not better off than a 3.6 who plans to go to [insert school here who I am not going to offend by naming], if both of their plans make total sense for their academic record and future plans. Academics are the least-important of the three criteria, as demonstrated by it only being scored on a 2-point scale, whereas leadership and service are scored on 3-point scales.

That said, as I wrote yesterday, there are a TON of very high-GPA students in the pool. I would guess that around 2/3 to 3/4 of the pool has a GPA of 3.9+, with almost all of the rest in the 3.7 range, and about 5% in the 3.5 range. So, yes, there are a lot of Truman finalists and scholars with sky-high GPAs, but that reflects the applicant pool, not a bias in the selection process. My team put forward several students in that 3.5-3.7 range. It's a remarkably strong pool academically -- and most of them have reasonable graduate school plans. Where students lost points with me was when their grad school plans and future career did not match what they have done in the past -- when that disconnect appears.

Lessons learned

I've been trying to think about "lessons learned from reading ~50 files" last night and this morning. It feels like most of what I have learned, actually, has been about the letter of nomination and about recruiting candidates. While writing the Truman application is a non-trivial undertaking, I think it's also really, really hard for a candidate to create a Truman application that falsely represents who they are in a positive way (though a poorly-done application may not positively represent them). Also, thanks to the copious amount of advice about the Truman application on their website and Tara's consistent presence at both the NAFA conferences and on the listserv, I think most of what I needed to learn about how to advise a Truman candidate I have picked up over the years -- I don't feel like there have been big, revelatory moments where I smack my head and say, "Oh, so THAT's what they are supposed to say in question 11!"

That said, here are some lessons learned about the letter of nomination and recruiting candidates:

Letters of nomination:
1) Get to the point. Most of my letters have been in the 1000-1200 word range. I will be aiming for 600-800 from now on. This doesn't have to be literature, and extraneous info makes me want to start skimming.
2) Introduce the candidate, tell us what they care about, and give us the highlights of what we will be reading.
3) Be upbeat, but don't oversell. I'm going to read the file myself, and get my own impression of the candidate. If you have set me up for greatness and I don't see that, it's disappointing. Save "She seems to be the ideal Truman candidate" for those who you have really good reason to be the ideal Truman candidates -- I have read several "ideal" files this weekend that ended up scoring out as a 2.5 and were nowhere close to being a finalist.
4) If you have a history of working with Trumans and can compare this candidate to past finalists/scholars, that makes an impact on the reader -- but do so sparingly, and only when the case is truly warranted.
5) Remember: we are looking for scholarship, leadership, and service. Stick to those. We don't really care too much about how funny he is, or how well she runs.
6) Think of it as Cliff Notes for the application. Give us the thesis, outline the story, and close with an argument for advancement. Then let the student do the heavy lifting in the application.

Recruiting. From now on, I am going to go seek out students who exhibit the following combination of traits:
1) Proven leaders who have shown initiative and gotten results
2) Public servants who have a demonstrated record of caring
3) Top students who have a demonstrated plan for the future (the second half of this is the one part of a Truman app that I think can be created, and where good advising may make the biggest difference)
4) Students to have an issue or idea that ties together their work and compels them forward
5) They should have some connection (minimum) to the government, public sector, or campaigns. More than that is welcome and encouraged, but often times not present.
6) Know that "outstanding" candidates (8+ points) are a rarity -- we have yet to see one in the ~70 files my team has scored so far. It's an interesting combination -- Truman kids are all amazing, and they could all use improvement. Almost every candidate has visible flaws, even those who I am near-certain will end up as Scholars. That's ok. Nominate them anyway.

Lastly, I think that we, as advisers, have a tremendous responsibility to help our candidates understand that this is not a referendum on them as people. I have one specific candidate in mind as I write this. She is an incredible young woman -- a top scholar at an impressive school, is well-respected, has some significant accomplishments, and her letters of recommendation are absolutely impassioned about who she is and how much they admire her ("top student I have known in my 35 years doing this" kind of letters). But she's not a Truman -- I think I scored her as a 3.5. Her record of leadership was not what we are looking for (no record of creating change) and she's been all over the map in terms of her activities. Those activities have been impressive, but have not been consistent. You can tell that she poured herself into this application, and I can only imagine that, based on her letter of nomination, her adviser is breathlessly awaiting her announcement as a finalist, which will never come.

As advisers, when we recruit and nominate the right students, we are setting them up for success in the competition. When we recruit and nominate the wrong students, it is still ok -- this is a bear of an application, and candidates learn really important lessons from wrestling with it. Look, that young woman is going to be perfectly fine in life -- more than "fine." She is going to go on to do great things. Just not with a Truman, and that's ok -- this is one model of leadership and social change, but not the only one. I hope her adviser tells her that today.

Friday, February 18, 2011

Perhaps you expected fireworks and confetti?



My reading team, doing what they do best. I was never sure what mental image to have of readers doing their work -- this is what it looks like. Three people, a stack of blue files & rating forms, some pencils, a bunch of coffee, a round table in a hotel room, and boats out the window. Get back to work!

Cuuuuute Baaaaaby



For all the FacReps who missed Tara in the fall while she was on maternity leave, here is her adorable excuse! Pascal made a triumphant visit to the hotel today, and took a break to sit on some files for a few minutes and entertain us.

Oh heck, YOU choose them.

Steel yourself: this will be a long blog post. It's been a VERY long day of reading -- 7:30am to 6:30pm, with an hour break for lunch. Our team read, scored, and made decisions on 53 files today, of which I read 32. That's not-quite pulling my weight, but one of the other readers on my team is a law school admissions person, so she reads files for a living and is pretty quick at it.

Lessons from today:

1) Questions 7 (leadership) and 8 (public policy) are really key, but 7 is the most important essay in the application to me (other readers might disagree with me on this). I see the questions as asking "what change have you created?" and "what do you care about and why?" If those aren't solid, there isn't a lot of case to be made for advancement.

2) The letter of nomination serves almost as a thesis for the application. It's hard to hide the ball -- all of the letters of nomination are glowing, but sometimes you finish one and notice that, gee, they never mentioned leadership. Then you read the file and, sure enough, there is no record of leadership to speak of. So you read them for what is in there, but also what is not in there.

3) I know most of the time if a candidate might advance in the first few pages. The record of achievement has to be present in Qs 1-6, it needs to be explicated in Qs 7-8, when the application turns to the future in Q 9 it needs to make sense, and Qs 11-14 need to be solid (of these, Q14 has the most potential to impact my opinion). By the time I get to the policy proposal, transcript and letters, I'm just looking to see confirmation of what I already know of the student, and don't read them as thoroughly as the rest of the application.

4) One of my fellow readers said, "Put this in your blog: in their career goals they need to be bold and say that they are going to have an impact. They shouldn't just want to have a job -- tell us what they will achieve in that job. They've been achieving things throughout college, so why won't they continue to achieve them in their careers? They should be ambitious."

5) LOTS of very high GPAs. I would estimate about half are in the 3.9+ range. Very few are <3.7. It's not that Truman has a stated preference for these insanely high GPAs, just that these are the candidates who are showing up on their doorstep.

Now! Details! With Tara's permission, I am going to give you a one- or two-sentence description of each file I read today, without identifying info, along with the score I gave, the other score given by my reading team (in parentheses), and then the outcome of that file at the end.

We started by finishing our region from yesterday. You'll recall that we had one BIG state left -- 30 files in it -- from which we could advance 8 as finalists. I read 19 of those files -- here are my summaries:

1) Passionate, thin record of service & leadership. Came to the passion late, through an internship. Well-done policy with thin sources. 4.0 (3.0)
2) Good kid who cares, but little record of being a change agent. 4.5 (4.5)
3) Activities all over the place - no focus - chosen field for grad degree has no coursework in BA. 3.5 (3.5)
4) Consistent. Some worries about public service commitment, but exceptional leadership skills & an excellent student. 6.5 (6.5)
5) Excellent student with very thin record of leadership & no vision of future as a change agent. 4.0 (4.0)
6) Excellent student, amazing leadership, interests are a little too broad, but clearly a finalist. 7.5 (9.0)
7) Not quite as strong a record of accomplishment, but a phenomenal story and someone you root for. 7.0 (7.5)
8) Borderline finalist. Showed excellent initiative and obviously cares, but focus on issue is overly broad & career path doesn't aim him at outcomes that will create an impact. 6.0 (5.5)
9) Came to college with one passion (not in service) and is bending that into a Truman app. Nothing impressive or convincing. 4.0 (5.0)
10) Personal history makes her passionate about chosen field, but it's not borne out in activities. Application shows a lack of attention to detail, and does not reveal thought and care. 3.5 (4.5)
11) Phenomenal student with no sense of direction,thin record of leadership, only short-term service 2.5 (5.0)
12) Good service, but nothing that popped out of leadership. Didn't see the impact of his work. Reasoning is compelling. 4.5 (7.0)
13) She cares a lot & has stutter-stepped through leadership activities, but I am not sure there is enough here. Her lack of clarity on domestic vs international future & lack of international experience (if she goes that way) is problematic. Close but not quite. 6.0 (7.0)
14) Terrific student with a very thin record of service and leadership. Very little non-academic. 4.0 (4.5, 5.5)
15) Great student who has great passion and some fascinating experiences, but seems to have a lack of results/follow-through. Right on the bubble. 6.0 (5.5)
16) Poor writing skills. leadership isn't creative and doesn't show initiative. Doesn't have a track record in career area. 2.0 (4.0)
17) Terrible start to application -- seems trivial. Picked up at end, but leadership example really hurt. 4.5 (4.0)
18) Average applicant. Poorly-done application. Q9 is total turnaround from earlier interests. 9 through policy is disconnected from rest of application 4.0 (4.5)
19) Lacks innovation & impact. Good passion. Has not achieved significantly, but has an excellent future plan. 4.5 (5.5)

So -- which ones would YOU take? I'll tell you that four of these advanced. Choose yours, then see if they match what we did...

4 and 6 were all easy and moved on as finalists without much discussion. So did three other files that I didn't read. 7 took a couple more minutes, but wasn't too hard. Then we had a big messy middle that included 8, 12, 13, and 15, five files from a state we had read yesterday, and three more from this state that I hadn't read. That's 12 files, and one spot left. About five minutes of conversation got us to 13 and 15 as the top two candidates, but with some disagreement over which was better -- though agreement that both were finalist-worthy. I liked 15 better, but my two reading partners both preferred 13, and I could see that I wasn't going to change their minds. 13 is a finalist, 15 isn't, and won't know how close he was.

Next region!

4 states. 23 files. State A only had 1 applicant; States B and C each had 4; State D had 15. We really want to have at least one finalist from each state, and two as possible (competition! choices!), so those from States A, B, and C were pretty lucky. And with getting to name up to 11 finalists, those in State D were pretty lucky, too. I didn't read the file from State A, but it was advanced by my co-readers.

State B:
20) Phenomenal student with great app (well-advised!). Weaker record of service and leadership. Immensely likable with an outstanding grad school plan. 6.0 (5.0)
21) Over-hyped application with a lot of titles and experiences, but no significant accomplishments or change. Outstanding dedication and passion. A finalist, though I don't care for it. 5.0 (6.5)

My prediction came true -- the others liked 21 much more than I did, and he advanced as the lone finalist from this state. We just couldn't make the case for advancing more than one from this state, unfortunately.

State C (forgot to note co-readers' scores):
22) LOVE her as a person, but she doesn't quite have the background. #14 was fantastic. I'm a big fan! 5.5
23) Very good record of university service, weaker service record, excellent academics and a good future plan with some vision. A solid finalist, but not blowing me away. 6.0
24) Interesting future plans, but not really tied to past, and without a solid record of service or leadership. Unimpressed, though academic record is very strong. 3.5

23 advanced pretty easily. The one other file in this region wasn't too impressive, and we really did want to advance two files...so 22 got through! The other reader on this file felt similarly -- just really, really rooting for this kid.

State D
25) Fantastic student, strong leadership, pretty good service -- all totally disconnected from future goals, and policy statement is on a 3rd topic. 5.5 (6.0)
26) Marvelous person with a great story, but record is thin and I question how much he has accomplished. 5.0 (5.0)
27) Not hugely impressive, but theme is good and future clear. She's set herself up nicely. 6.5 (5.5)
28) No connecting thread in application. Very poor Q14. Good political experience, but that's about it. Tepid letters of rec. 3.5 (3.0)
29) Doesn't hang together. Lots of issues he likes. No direction. Fantastic letters. 5.0 (6.0)
30) Nice kid with an interesting plan, which is disconnected from past. No real direction. Leadership example very weak. 3.0 (2.5)
31) Important role in campus org & has good political experience, but goals are all over the map and very vague. No real reason behind grad school plan. Terrific letters. 5.0 (4.5)
32) Solid app, lots to like here, not over-the-top, but feasible candidate who is diligent & no-nonsense. Mature. Has potential to make quiet but important impact. 6.5 (6.0)

We could name up to seven from this state, but were pretty sure from from the start that we didn't have that many in the pool of 15. 25 and 27 were named as finalists pretty quickly, along with two others I didn't read. 26 and 32 came under serious consideration, along with two other files I hadn't read, but skimmed through enough to have some opinions on. The more we read 26, the more we liked him, and he advanced. That left 32 and the other two, which we debated for about 10 minutes. One of the other two fell away. Should we name both of these final two? We had the space. In the end, 32 was just a solid, solid candidate in ways the other one wasn't, so he advanced as a finalist.

So that's what it is like to read, score, and decide on 32 files in a day. 11 of those 32 became finalists, and I would guess that about five of them will be named as Truman Scholars.

Now: dinner, read (not files!), bed, and our last region in the morning.

The blog post that isn't

This is a quick post to say there is no post. The reading process is more intense than I anticipated, and the set-up does not lend itself to breaks to write a blog. We have just finished our first region and decided on our 11 finalists, which feels good. I am taking copious notes and look forward to sharing them with you tonight.

Tara says that the first notifications will go out tonight -- emails to finalists and a change in status on their application if FacReps log-in and look at their registered candidates.

Also: Maryland crab cakes are delicious.