HRMDirect President Colin Kingsbury writes on the latest in recruiting and technology.

Enjoy the blog? Wait until you try our applicant tracking system. Click for your 30-day free trial.

About a month ago I posted why the typical software-vendor ROI story should be filed under "fiction" and signed off by threatening to go after the RFP process next.

Along the way to writing this I experienced one of those moments of clarity where all the complex and overlapping ideas I'd started out with just sort of fell away, leaving behind a single explanation, elegant in its simplicity:

RFPs are like job descriptions, and RFP responses are like resumes

Every day HR departments terminate employees whose resumes met every stated requirement on the job description. In many cases, the job description and the resumes it attracts all deserve to be filed under "Fiction."

As a buyer, the purpose of an RFP process is to allow you to differentiate between a selection of products to solve a particular need. By giving everyone a list of standardized questions, you're able to make decisions on a more objective basis. (If you work for one of our competitors, you can stop laughing now and just email me a copy of your resume--it's ckingsbury@hrmdirect.com.) Problem is, that's just not how it works.

The typical RFP either asks the vendor to respond to a series of "can your product do X" questions, or, in some cases, asks them to rate themselves on a scale of 1-5 or some such. About a year ago in When Bad Features Feel Good I wrote, "Successful vendors are successful because they build products that people buy. That's obvious but what's more often ignored is that people often don't buy the 'best' product objectively speaking." So let's add another item to the list:

Successful vendors are successful because they're better than everyone else at convincing people to buy their product.

Just as a person putting "Java" on their resume serves as no guarantee that they are any good at programming in it, a vendor giving a positive answer on an RFP serves as no assurance that you'll actually like the product. You might think your RFP is so cleverly-written that we'll actually be forced into giving candid, clear responses--but remember, two can play at this game, and vendors get a lot more practice. After all, 80% of people think they're in the top 30% in terms of driving ability.

My advice is to look at the RFP for what it is: a prenuptial agreement that should be taken seriously by no one except the purchasing department. It will not help you to differentiate in terms of vendors' abilities to deliver a satisfactory solution. It will help to differentiate between those vendors who have large sales departments and/or teams of proposal writers to respond to every RFP that hits their inbox and pass that cost along to you. Salesforce.com is illustrative in this regard: their most recent annual report shows that in 2006, they spent 5 times as much on marketing and sales as they did on research and development. This represents an improvement actually, considering that the gap in 2005 was a factor of ten.

Just as requiring a Master's degree for a job which doesn't require one can have an adverse impact on your applicant pool, sending an unqualified, 30-page RFP out to two dozen vendors pretty much ensures that the responses you get will be composed mostly of the costly and the desperate.

[ add comment ]   |  [ 0 trackbacks ]
permalink  |   ( 3 / 572 ) Martin Snyder has never made any secret of his distaste for the OFCCP Internet Applicant rules, and in his most recent blog , he lays into a BusinessWeek article which basically says, "Don't hate the OFCCP, hate your lack of process discipline." Martin writes,

I agree, but I remain hopeful that at some point soon, enforcement action will commence and this matter will end up in federal court, where right thinking jurists will see it for what it really is; a constitutional affront, a violation of Executive Order 12866, and a costly mandate that neither meets its goals nor creates better processes.

How do you feel, really? Later in a comment, Martin adds,

This meme that the rule is somehow like SOX is bad spin.

Don't worry about me giving up beating on the rule- its my white whale!

Moby Dick happens to be my favorite book, one of the few I've read repeatedly over the years, and Martin, I hate to break this to you, but the whale wins.

CFOs aren't widely known for their acute sense of wit, but if there was such a thing as the annual CFO comedy awards, Martin's line that the OFCCP regs are not comparable to Sarbanes-Oxley because "SOX is designed to give quantitatively better accounting" would bring the house down. Many would, I suspect, say that SOX served mostly the same purpose as the tails on senators' tuxedo jackets. Martin is welcome and may well be right to say that the OFCCP rules are attacking forms of discrimination which are so imperceptibly small as to effectively not exist, but that's really a different argument altogether.

Chad Sowash of Direct Employers (one of the more important organizations out there today, IMHO), has in my mind a better response in his post, which mostly echoes the central point of the BusinessWeek article: consider the Internet Applicant rules as an opportunity to do a lot of things better.

In my experience, HR and recruiting departments, particularly in smaller companies, are often led by people with good intentions but who are gasping for organizational resources and attention. For many of them, the necessity of the OFCCP rules provide a convenient anvil not just to achieve compliance, but to optimize actual recruiting results.

[ add comment ]   |  [ 0 trackbacks ]
permalink  |   ( 2.9 / 640 ) Double Dubs at Systematic HR has post today asking whether Google's much-ballyhooed algorithm for recruitment is really old fish wrapped in new paper. I've studied a lot more statistics than most people, and they've made me more skeptical than anything else of these types of tools.

Statistical analyses are kind of like recipes: the more variables you put in the mix, the more opportunities there are to mess things up. If you show me a regression with more than 2 or 3 variables and can't explain what heteroskedasticity* (for instance) means, I'm going to start getting skeptical about your forecasting technique. Howard's comment on the post linked above with its example of "hundreds of variables" makes me wince. Especially if this was in the 80s, when computing time was much dearer--today you can do in a few minutes what might have taken hours, so you're more likely to try lots of scenarios and models before accepting the conclusions.

The real problem, as always in this game, is the choice of variables. A successful retail cashier, for instance, can probably be defined very well in terms of integrity, accuracy, diligence, and attitude. If you generally show up on time, don't steal, treat customers nicely when you're haing a bad day, and count change carefully, then you're doing great. With a population of hundreds of thousands of workers to survey, a good predictive assessment is very attainable.

As jobs become more specialized in terms of skills and knowledge, it becomes harder to build a representative sample pool from which reliable assumptions can be drawn. While large companies theoretically have an advantage here, my experience has been that organizational opacity increases with size, and determining what and who generates value becomes harder.

Evidence for this comes from the widely-shared sense of futility around the traditional performance review process. Forget about recruiting--many companies still have a very hard time quantifying the value of an employee just crossing her one-year anniversary, and where she will be in 12, 24, or 36 months. Perhaps once a company has that part down reasonably well they can start thinking about predicting success among people they barely know.


* Can't help yourself? Here goes: Heteroskedasticity is actually more complicated to spell than it is to understand at a basic level. A great example is looking at education versus income, as seen on this chart. Going from a HS diploma to a 4-year degree is huge, but the incremental improvement at each step beyond is less obvious (especially relative to the time/money investment). Most notable is that a Ph.D. (which takes 4-8 years) is worth less than a professional degree which requires between 2 and 4.

The reason for this is that as you move farther up the education ladder, the effects on career become more complicated. While a Bachelor's degree opens up a world of opportunities, most of which are better-paying, many of the people earning higher degrees are doing so not to make more money, but in order to access jobs (like a university professorship) which are rewarding in completely different terms, and in many cases actually worse-paying. This is "heteroskedastic error" in a nutshell.

In order to accurately assess the effect of years of education on income, we need to control for this variation either by focusing on one type of job, or by using a more sophisticated modeling approach that controls for this type of variation. As it happens, the solution to this problem earned the economist Robert Engle a Nobel about five years ago.

[ add comment ]   |  [ 0 trackbacks ]
permalink  |   ( 2.9 / 629 ) It is thoughtful of Jeff Hunter to begin titling his recent post "Reqs are a Reality," because there's no shortage of recruit-o-pundits writing "why you must get rid of (insert buzzword here) or perish!" screeds. The pattern is so consistent that they're like a bunch of students who all bought the same term paper for the same class.

Jeff's post is important because nearly all the consultants, thought leaders, and arm-wavers, he's starting to put money and mouth together in his company. People who dismiss him as a starry-eyed theoretician will be disappointed, and if anyone is going to trailblaze a new model for talent over the next 5 years, Jeff's the man I'd bet on.

That said, I remain broadly skeptical. Jeff writes,

Regardless of any of that, reqs are guilty of all the sins I have enumerated so many times before: the are a risk control document that means more to the CFO than the talent, they lead you to measure yourselves by speed instead of value, and they drive clients to see you as a cost / risk problem in the organization instead of a critical business value add. None of that is in dispute. I still haven’t had anyone give me a rejoinder that would make me believe that I had any of my facts wrong.

This is, like everything else Jeff writes, a sparkling polemic. He also to my mind risks giving the game away in the first sentence.

Mommy, where do baby reqs come from?
Saying that "reqs are a risk control document that means more to the CFO than the talent" suggests that the CFO is some kind of green-eyeshaded troll blocking the bridge to a halcyon future of good cheer and profit. Until talent (not to mention landlords and suppliers) are willing to accept payment in "talent" rather than "cash," it is the organization's consolidated financials which determine possibility.

At a more atomic level, reqs represent the overall direction and plan of the company. Where I am most sympathetic to Jeff's position is that many companies approach this so reactively that even within the strictures of "we can afford to spend $500k hiring people in this department in 2007" there is no doubt room for improvement. But, I think talking about this primarily from a recruiting or even HR perspective is problematic to say the least. We've all seen, known, or been one of those people hired to "change the dynamic" in a department or company, and in the absence of a much larger directional transformation, such people rarely succeed.

Jeff's uncompromising vision does not demand a better kind of recruiting; it demands an entirely different kind of business. In that I wish him good luck, and godspeed.



[ add comment ]   |  [ 1 trackbacks ]
permalink  |   ( 2.9 / 1049 ) There's currently a thread on ERE asking how one goes about calculating the ROI on an ATS. You can replace "ATS" with any other business software acronym here and it's the same story pretty much: we can help you shave a nickel off every $5 bill that passes by, and at the end of the year you'll have this jar full of nickels.

Perhaps the best put-down of this I ever received was some years back when I was an SE presenting a knowledge management product to a leading manufacturer of power-generation equipment that would have helped their techs find answers and relavant documentation much faster than they could currently. They had about 8,000 techs, so these were big numbers we could work with. We ran tests that showed techs spent an hour or so looking things up each day, and with our system, they could find the same info in 30 minutes or less. Multiple .5 hours times 8000 people times $50 per hour cost and we could show a positive ROI in one day.

The decision-maker never argued with our basic assumptions. He just said, "So they save 20 or 30 minutes? They'll just take an extra cigarette break." Far from being flippant, the VP was making a really important point. You can definitely get the pennies and nickels out of the process, but they have a habit of disappearing on their way to the jar.

ROI analyses work well when the dynamics of the system are very well known. For instance, the ROI on replacing a dirt-cheap inkjet printer with even a moderately-expensive laser printer is a no-brainer if you do much printing, because the laser doesn't guzzle ink that costs more than first-growth Burgundy by volume. You can easily plot a graph that shows that after X pages, the laser will be putting money back in your pocket. The variables are easily identified and quantified.

Looking at the ATS business, there is a major gap between cost and value. We charge $1200 per year per recruiter using our product, and we know from our clients that the vast majority of users spend nearly their entire working day staring at our system. Many of those users will cost $100k or more in salary and benefits, and they use our system because it makes them vastly more productive. How do we know? Because not a single client whose subscription was up for renewal has ever cancelled because "we realized we really don't need an ATS after all." A huge chunk of our business comes when a new head of recruiting or HR joins a company that doesn't have an ATS and says, "you gotta be kidding me!"

How do you quantify that value? Time-to-hire may not be good because the time saved by the ATS may accrue to recruiters spending more time sourcing or screening, to give just one example. There are a lot of moving parts, but I think it's safe to say that if ATS's were as anti-productive as they are often made out to be, there wouldn't be such a strong market for them that a company like HRMDirect could add 70 clients in the past 12 months without spending Jobsterbucks on marketing. But because we can't pin the numbers down the way we can an inkjet-vs-laser, we end up charging what the buyers are able to pay. CRM systems, which in many cases provide a similar set of functionality to our ATS, often sell for much more. Why? Because they're often bought by sales departments, which have far better access to budget.

In my experience, the sort of ROI analysis attached to most software buying decisions is made after the vendor has been selected, which is to say after it really matters. More often than not, the analytical model used to calculate the ROI was provided by said vendor, and is about as objective as asking a Red Sox fan what she thinks of this year's Yankees. But then, most buyers are by this point in cahoots with the vendor, and care about the ROI case only to the degree that it provides posterior insurance should the decision prove unwise*.

And to be entirely fair to all involved, often this is just a case of the buyer complying with internal bureaucracy. I wonder sometimes whether the finance departments that ask for this stuff really look over it carefully each year, or whether everyone except the shareholders is in on the joke.

All of this isn't by any means to say that thinking about things in terms of ROI is fundamentally misguided. But the systems by which we approach this today are for the most part barely nicking the surface of what's really going on. What I do know is that once a recruiter starts using a tool like Resume Direct, it becomes an indispensable if unglamorous part of their job, for less than most of them spend at Starbucks on their way into the office. Maybe our applicant tracking system is really that much better, or maybe people just like to complain, but I don't think I'm hallucinating.

If there's any good news in all this, it's that I'm being asked to provide these sorts of ROI studies far less often than 2 or 5 years ago. The once-ubiquitous "ROI calculators" (which are about as serious a forecasting tool as a magic 8-ball) no longer occupy prime billing on vendor websites. I'll take progress where I can get it.

* And therein lies my next post, which will be about ROI's evil degenerate twin, the dreaded RFI.

[ add comment ]   |  [ 0 trackbacks ]
permalink  |   ( 2.9 / 936 )
Back Next