Wow! I was fortunate enough to present at the family reunion of sourcers everywhere – SourceCon Spring 2017. (Yay!) At least, that’s what it felt like to me. But I digress, I shared the stage with Jason Roberts who co-presented with me “Human vs. Machine.” Unless you are new to SourceCon and its ways, we produce contests that test the mettle of Sourcerors, Purple Squirrel Hunters, Unicorn Riders, Sourcing Ninjas, Searchologists and Boolean Bandits the world over. The contest is always tricky and requires both logic and creativity to conquer. All of the contests this round lead up to the ultimate competition (insert dramatic opera style music with angels singing here)… Human vs. Machine.
Do you hear it? Boom! Ka-boom! Echoing Thunder!
Okay, so maybe you don’t hear it; the contest was dramatic and exciting nonetheless. Eight contestants… (No, that’s not right.) Eight of the web’s research warriors fresh from battling and defeating obstacles that would rob the joy and raison d’etre of lesser sourcers vowed to conquer the Grandmaster Challenge, for a lifetime of SourceCon glory! (Insert more dramatic opera style music with angels singing here)
The quest for immortality was simple enough:
- Download a folder of three jobs: Ground Service Agent, Systems Administrator, and Product Manager. The jobs were real but altered from when they were originally posted.
- Download a trove of over 5,500 resumes.
- Search thousands of resumes and find the people who were hired, interviewed and sourced for the roles; by an undisclosed company.
- Points were given when the right resumes were found and classified correctly. (i.e. This person was hired. This individual was interviewed. Et cetera.) Points were also given if contestants found the right resume but categorized it incorrectly.
While the humans were searching, an artificial intelligence from Brilent was also playing the game and playing to win. The humans spent from 4 – 25 hours to research and submit their entries whereas the machine took 3.2 seconds to deliver its results. In the end, the humans won with the machine coming in 3rd place! Guillaume Alexandre and Sarah Goldberg were tied for second place with Randy Bailey taking the coveted title of SourceCon Grandmaster.
Here are some observations and quotes from the winners:
3rd Place – Brilent
- Found the Sys Admin hire and all the candidates sourced for it.
- Found the candidates originally sourced for the Ground Service Agent position.
- None of the resumes we intended were found for the Product Manager role, but those sourced were compelling.
2nd Place (Tie) – Sarah Goldberg
I organized everything into one folder on my desktop and just used boolean searching within Windows Explorer to narrow things down. First I got rid of a bunch of extra files – people applying for recruiter jobs, account managers, customer service, etc. Then I figured out (or tried to, at any rate) the real companies and locations behind each posting, and what kind of profiles succeeded there. I copied likely files into a separate folder (one for each role). From there, I narrowed every role down to about 30 top candidates and went through those resumes individually, and then trusted my gut for the top three. After I had settled on my strategy, I think I spent probably four to six hours running searches, researching the jobs, etc.
2nd Place (Tie) – Guillaume Alexandre
This was my approach to the challenge with the information provided. In a challenge where it was human vs. machines, I decided to focus only on the provided data and work two ways.
Step 1 (machine phase) Reduce the amount of supplied data to a “human level.” I used Docfetcher and Copernic, two tools that would enable me to do a boolean search on the provided 6k+ CVs. I did A LOT of different strings to try to extract all the potentially useful data and narrow it down to about 10 CVs per position.
Step 2 (human phase) I decided to use the oldest tool a sourcer can use, its eyes, so I physically printed all the CVs (which I normally NEVER do) and took a highlighter and started to read every line of every CV to find the ones that I liked best and made an “educated guess.”
My main struggle was about making a decision with so few info. When you don’t know the location, the size of the company you’re recruiting for, its industry, the number of servers for a system admin role, it’s MUCH harder to chose CVs. Questions like “do I chose a ground agent based in California or a guy who hasn’t done it, could do the role but is based nearby” I read the job descriptions hundreds of time trying to paint myself a picture of the company (like a “persona” but the other way around). With this in mind, decide, as a human, and then see if the machine had the same results (once again assuming the machine had the same data as I did and nothing else).
I did find out very quickly that some names, addresses, and companies were changed in some CVs (the author has been modified to Jim Stroud for example, and the guy was living in candy cane lane or other things) but deliberately chose to ignore this potentially valuable information as, to me, these CVs were not the ones matching the best the job description with the info I had in hand.
Grandmaster – Randy Bailey
System Admin: One of the first things I did was put the SysAdmin job description into google, found [the company], shortly after that found [the candidate who was hired]. I was confused when I couldn’t find his resume. Eventually, I searched for the keywords of Bakersfield & the school, finding the real resume, which matched up perfectly, except for altered contact, company names to that of XXXX. The runners up were a little more challenging and required doing searches for a variety of keywords in the resumes. The runner-up, who I eventually confirmed, was found through a variety of keyword searches. My sourced person also came up through a variety of keyword searches and also was confirmed as a close match using Hiring Solved.
Product Manager: Had trouble finding the real job description for this, but did a variety of keyword searches that eventually lead to XXXX’s resume (42788_99198). From that resume, I eventually was able to find XXXX’s profile and could see she was at XXXX. I still had trouble finding a JD that fit, partially because, without CEB TalentNeuron, it’s tough to find archived resumes. I eventually found I could use Archive.org and found a resume that was close-ish had the appropriate keywords and covered the gist of the job. My runner up (person interviewed) was XXXX (real name: XXX). I eventually reached out to XXX, as well as the others, but she confirmed that she had interviewed with XXXX back in 2015. (thus confirming this was probably the correct JD). I am less confident about my sourced person, XXXXX, (real: XXXXXXX), although this was also one of the top people that Hiring Solved picked and went with it because the threshold was just someone their sources should have sourced.
Ground Agents: This was tough, primarily because it was the hardest to verify anything and the requirements were a high school diploma, right attitude, some customer service experience and being able to lift 50 lbs. I struggled finding any of the real people for several days, eventually going back to the resume and noticing that the “author” of the documents were “XXXX’s” for my person who got hired, XXXX (real name: XXXX, 6181_14299) and my person who they probably interviewed: XXXX (Real name & author of the resume: XXX).
I wasn’t 100% sure which to use as my top person, but my gut said XXXX and also other data suggested that as well. After searching every which way I could think to find some professional information for these two, I finally decided based on their connections. So XXXX’s mother is also a Ground Agent, and he was friends with a bunch of ground agent/airport employees. XXXX didn’t have the same network, only a couple friends that did anything with the airport including they were both connected to at least one person. I am just thinking about this now, that person worked at the airport as well, so may have been involved in the interview process? I also discovered that XXXX seemed to have copied parts of his resume from standard resume templates for his roles, but didn’t think that mattered as much. One thing that did give me pause was his twitter history talked quite a lot about getting high, which would certainly be a red flag for the aviation industry. I put him as the person to get interviewed because I didn’t think an RPO would go that deep at that stage of the process.
Some tactics used by other contestants included:
- Uploading the resumes into Evernote and searching them that way.
- Searching the meta code in the WORD documents of the resumes to see which were altered by other Randstad Sourceright employees or me
- Creating an algorithm in a Python environment to find the right candidate
Using search tools like Copernic, Docfetcher, Hello Talent and the search functions standard on their computers.
-
- Overall, the humans used software to whittle down their search to a manageable size then trusted their gut for the rest. In many cases, printing off the more interesting resumes and highlighting words with a yellow highlighter.
- Although the machine was 3rd in the contest, it competed well against some of the best sourcers in the world, and it only took a few seconds to do so.
- Many people fear the uprising of machines but, maybe those who are most afraid should embrace them. After all, it was a combination of humans and machine technology together that enabled the humans to defeat the artificial intelligence in the competition. Hmm, maybe the next SourceCon competition should be cyborg vs. machine? I have a feeling the cyborgs will be unstoppable.
(Insert more dramatic opera style music with angels singing here)
See you at the next SourceCon!
