Follow me on a journey… a journey of bad data, stretched conclusions, and mysterious results.

On Sept. 18th, Kaplan released a survey (remember this one?) that “at ‘top schools’, one out of ten admissions officers has visited an applicant’s social networking Web site as part of the admissions decision-making process.” The survey was conducted with a whopping 320 admission counselors.  [Link]

I caught the story on Sept. 19th when the Chronicle Wired Campus posted the results [Link].  They state that “One in 10 admissions officers has looked at an applicant’s social-networking profile”, which is a much broader statement than the original survey. Kaplan notes that they looked at an applicant’s social-networking profile as part of the admissions decision-making process. Reporting Fail #1.

And here’s Reporting Fail #2: The Chronicle article states that “The company surveyed 320 institutions among U.S. News & World Report’s and Barron’s top 500.”  Look back at the Kaplan article to see that the methodology “for the 2008 survey, 320 admissions officers from the nation’s top 500 schools – as compiled from U.S. News & World Report’s “Ultimate College Directory” and Barron’s Profiles of American Colleges – were surveyed by telephone between July-Aug 2008.”

So unless the Chronicle can prove or reasonably assume that they only surveyed 1 worker at each school, this statement is incorrect and unreasonably stretches the data across a wider sample.  We have 10 counselors at Butler. Kaplan could have called 32 schools and interviewed 10 people at each.  We don’t know, because it does not say.  But what we do know is that there is not a possible way to interview someone from all 500 schools when only 320 people were interviewed.

I admit that I should have clicked back to the original Kaplan press release to read more, but I took the Chronicle post for what it was worth, and commented “10% of counselors? Hardly an issue. Most of those who looked were probably only there because the student requested to be their friend.”  I can think of several instances where my co-workers have had prospective students friend them on Facebook, myself included.  And most of the time, I look at their profile to see who they are.

Fast-forward to September 21st on Slashdot, which a member reports “10 Percent of Colleges Check Applicants’ Social Profiles” [Link].  The schools involved are now only “prestigious” ones.  Following this article is a very heated discussion about this, over 300 comments at the time of this writing.

So we’ve gone from 32 out of 320 admission officers saying they have looked at a social networking profile of an applicant as a part of the admissions decision-making process, to 320 institutions being surveyed and 10% saying that they look at social networking profiles, to now…. 10% of colleges checking social profiles.

I’ve given the office a pretty basic explanation of how social media fits into the admission process. If you do it for one, you must do it for all.   And since you can’t do it for all, then just don’t do it.   Seems to work fine so far.  But when a student reaches out to be my friend on Facebook, then I friend them. And they usually ask me questions, because that is how they communicate.   It’s probably easier for me since I don’t read apps or make decisions, but I know our staff does a great job at evaluating the applicant the same as everyone else, and based solely on the materials included in the app.

How does your school handle all of this?