August 7, 2015

Talent, Tools, and Time: Why So Few Companies Act on Big Data

DangleIn a recent survey, Scout Exchange found that 57 percent of HR professionals “felt that using big data principles was better than relying on their ‘gut'” — and that only 16 percent of these same HR professionals were actually using big data to review hiring trends.

According to Scout Exchange CEO Ken Lazarus, there are a few reasons for this disconnect — not the least of which is the fact that, despite all the breathless hyperbole, the recruiting industry is really only taking its first steps toward big data.

“It took a long time [for HR] to get to that point where so many people actually think [big data] is useful,” Lazarus says. “That was a long process — maybe it’s taken 10 years or more to get to a metric- and data-driven environment in HR.”

And now that we’ve realized big data can be useful, we need to figure out exactly how to use it. That itself will take some additional time.

Lazarus also notes that many of the same people vociferously advocating for big data approaches to HR don’t totally understand what big data actually is.

“A lot of the hyperbole is confusion over what big data even is. That’s part of the issue: people need to get more educated on what’s traditional analytics and what’s big data,” Lazarus says. “Some people will say, ‘If I have a lot of data and I’m using analytic tools, then I’m using big data.’ What they’re really doing is taking their first steps toward using analytics to make business decisions — and that’s fine, but it doesn’t mean it’s really big data.”

Lazarus defines big data as “data so big, it’s hard to get useful insights out of it. You need algorithms that can find the insights for you, or super experts who can slice up the data in ways that make it useful.”

As an example of big data in action, Lazarus refers to recent developments in cancer research, in which researchers are studying millions of tissue samples — each one containing millions of cells — and attempting to predict with ones will become cancerous.

“If you’re looking at all these samples, it’s so much data that it’s mind-boggling,” Lazarus says. “Big data analytics can [use algorithms to] sort through that and find the underlying trends.”

Such research is a far cry from what passes for big data in HR today. Analyzing millions and millions of cells is dramatically different from, say, tracking how many applicants you receive through a given job board.

Talent, Tools, and Time: Three Obstacles to Adopting Big Data Principles in HR

As HR and recruiting stand on the cusp of big data, three major challenges keep them from moving forward: a lack of talent, relatively primitive tools, and heavy workloads that sap time.

1. Talent: Data Scientists Are Hard to Find

“To actually analyze big data and use the tools properly, companies need data scientists,” Lazarus says. “They’re actually really hard to find and in very high demand.”

RockMoreover, Lazarus says, companies can’t simply solve this problem by training their existing HR and recruiting professionals to be data scientists.

“You need the right background, the right math skills and analytics knowledge,” Lazarus explains. “[Data science] is a serious discipline; it’s not like running a spreadsheet.”

That being said, Lazarus believes companies can find the data scientists they need — it’s just going to be very difficult, much like the current war for software developers.

“Everyone has a hard time finding developers, but there’s still a lot of software being developed,” Lazarus says. “People figure out how, but it’s tough.”

One potential solution, Lazarus says, is hiring outside consultants instead of bringing full-time data scientists onto your staff.

Lazarus also believes that the evolution of big data tools may make data scientists slightly less critical.

“As more tools mature and become more user-friendly, the level of talent required on the data scientists’ side is going to go down,” Lazarus says. “Maybe, ultimately, we can take some of the more analytically minded folks in HR and actually train them to use these tools. But right now, the tools are too immature to do that. You really need someone with a lot of deep knowledge.”

2. Tools: Today’s Big Data Tools Still Need Time to Mature

“The technology infrastructure to actually use big data is still in the beginning stages right now,” Lazarus says. “It’s going to take time. We’re just scratching the surface with these tools right now.”

For example, in a recent informal survey of the hottest big data startups today, Lazarus found that the vast majority of these organizations are working on some “pretty mundane stuff.” Lazarus estimates that about one-third of today’s big data startups are working on streaming data; another third is working on pulling together different databases; and a final third is working on how to access big databases.

“They’re doing some cool stuff, but it’s pretty basic,” Lazarus says. “That’s an indication of how far we have to go.”

3. Time: Recruiters Have Reqs to Fill

“A lot of [recruiters and HR professionals] are just trying to keep their heads above water — trying to fill the damn reqs they have on their plates,” Lazarus says. “They’re getting a lot of pressure from hiring managers, and they don’t have a lot of time to be doing analytics.”

BenchTo solve this problem, Lazarus says, we may need to head straight to executive leadership. Companies like to say talent is their most important asset, but if they want to put their money where their mouthes are, they need to allocate resources to talent — and that includes giving HR and recruiting departments the time and funding they need to adopt big data.

“Maybe the HR execs need to argue for it, and the CEOs and others need to realize how important it is,” Lazarus says. “If you really think talent is your most important asset, then you better put some resources into data analytics in your HR groups.”

We’ll Get There — It’ll Just Be a While

Despite all these obstacles and misconceptions, Lazarus says he’s optimistic that, in the future, most companies will see to it that their HR departments have the time, talent, and tools they need to leverage big data in useful ways.

That being said, Lazarus also cautions that he’s “realistic in terms of how long it will take.”

“It’ll take a while,” he says. “We’re talking a timescale of a decade or so. Maybe 10 years from now, everyone will have these big data tools and analytics and actually use them to be much more effective at recruiting.”

For now, however, Lazarus doesn’t want us to jump the gun. It took about 15 years for PCs to affect productivity in HR departments, he reminds us, and about a decade before the Internet really “turned into huge productivity gains.”

Read more in HR

Matthew Kosinski is the managing editor of