Phrases matter: AI can predict salaries based mostly on the textual content of on-line job postings

The job panorama in the USA is dramatically shifting: The COVID-19 pandemic has redefined important work and moved staff out of the workplace. New applied sciences are remodeling the character of many occupations. Globalization continues to push jobs to new places. And local weather change issues are including jobs within the various power sector whereas reducing them from the fossil gasoline trade. 

Amid this office turmoil, staff, in addition to employers and policymakers, may benefit from understanding which job traits result in greater wages and mobility, says Sarah Bana, a postdoctoral fellow at Stanford’s Digital Financial system Lab, a part of the Stanford Institute for Human-Centered Synthetic Intelligence. And, she notes, there now exists a big dataset which may assist present that understanding: the textual content of thousands and thousands of on-line job postings. 

“On-line information gives us with an incredible alternative to measure what issues,” she says.

Certainly, utilizing synthetic intelligence (AI) and machine studying, Bana not too long ago confirmed that the phrases utilized in a dataset of multiple million on-line job postings clarify 87% of the variation in salaries throughout an unlimited proportion of the labor market. It’s the primary work to make use of such a big dataset of postings and to take a look at the connection between postings and salaries. 

Bana additionally experimented with injecting new textual content – including a ability certificates, for instance – into related job listings to see how these phrases modified the wage prediction.

“It seems that we are able to use the textual content of job listings to judge the salary-relevant traits of jobs in close-to actual time,” Bana says. “This data might make making use of for jobs extra clear and enhance our strategy to workforce training and coaching.”

An AI dataset of 1 million job postings 

To investigate how the textual content of on-line job postings pertains to salaries, Bana obtained multiple million pre-pandemic job postings from Greenwich.HR, which aggregates thousands and thousands of job postings from on-line job board platforms. 

She then used BERT, some of the superior pure language processing (NLP) fashions accessible, to coach an NLP mannequin utilizing the textual content of greater than 800,000 of the job postings and their related wage information. When she examined the mannequin utilizing the remaining 200,000 job listings, it precisely predicted the related salaries 87% of the time. By comparability, utilizing solely the job postings’ job titles and geographic places yielded correct predictions simply 69% of the time.

In follow-up work, Bana will try and characterize the contribution of varied phrases to the wage prediction. “Ideally, we’ll shade phrases inside postings from crimson to inexperienced, the place the darker crimson phrases are linked with decrease wage and the darker inexperienced are linked with greater wage,” she says. 

The worth of upskilling: A text-injection experiment

To establish which expertise matter for wage prediction, Bana used a text-injection strategy: To sure related job postings, she added quick phrases indicating the job requires a specific profession certification, resembling these listed in Certainly.com’s 10 In-Demand Profession Certifications (And How To Obtain Them). Acquiring these certifications might be expensive, with costs starting from about $225 to about $2,000. However, till now, there was no solution to decide whether or not the funding is worth it from a wage standpoint. 

Bana’s experiment revealed that some certifications (such because the IIBA Agile Evaluation Certification) produce significant wage positive factors rapidly whereas others (such because the Cisco Licensed Internetwork Skilled) achieve this extra slowly – useful data for staff who want to have higher details about how an funding in expertise coaching will have an effect on their salaries and prospects, Bana says.

Staff aren’t the one ones to learn from this data, Bana notes. Employers can use these outcomes to raised put money into human capital, she says. If, for instance, machine studying fashions reveal a gradual shift away from some duties and towards others, employers would have advance warning and will retrain sure staff.

And policymakers contemplating what job coaching applications to advertise would equally profit from understanding which expertise are waxing or waning in financial worth.

To that finish, Bana and her colleagues are at present engaged on a companion paper that identifies what duties are disappearing from job listings over time and what new duties are showing. 

Sooner or later, Bana hopes that textual evaluation of job postings might yield a web-based utility the place staff or firms might analysis the worth added by upskilling or by transferring to a brand new geographic location. 

“At the moment there’s not loads of readability round a path to greater earnings,” Bana says. “Instruments like these might assist job seekers enhance their job prospects, employers develop their workforces, and policymakers reply to instant modifications within the economic system.”

Katharine Miller is a contributing author for the Stanford Institute for Human-Centered AI.

This story initially appeared on Hai.stanford.edu. Copyright 2022