[US] AI can predict salaries using text of online job ads

The post-pandemic US job landscape is continuing to evolve, globalisation is continuing to push jobs to new locations and new technologies are transforming the nature of many occupations. In the face of such workplace turmoil, workers, employers and policymakers, could benefit from understanding which job characteristics lead to higher wages and mobility, VentureBeat reports.

Sarah Bana - a postdoctoral fellow at Stanford’s Digital Economy Lab, part of the Stanford Institute for Human-Centered Artificial Intelligence - is working towards exactly that. Ms Bana says a large dataset exists that might help to provide that understanding: the text of millions of online job postings. 

“Online data provides us with a tremendous opportunity to measure what matters,” she said.

Using artificial intelligence (AI) and machine learning, Ms Bana recently demonstrated that the words used in a dataset of more than one million online job postings explain 87 per cent of the variation in salaries across a vast proportion of the labour market. 

This is reportedly the first work to use such a large dataset of postings and to explore the relationship between postings and salaries. 

Ms Bana also experimented with injecting new text – adding a skill certificate, for example – into relevant job listings to see how these words changed the salary prediction.

“It turns out that we can use the text of job listings to evaluate the salary-relevant characteristics of jobs in close-to real time,” Ms Bana said. “This information could make applying for jobs more transparent and improve our approach to workforce education and training.”

An AI dataset of one million job postings 

In order to analyse how the text of online job postings relates to salaries, Ms Bana obtained more than one million pre-pandemic job postings from Greenwich.HR, which aggregates millions of job postings from online job board platforms. 

She then used BERT - one of the most advanced natural language processing (NLP) models available - to train an NLP model using the text of more than 800,000 of the job postings and their associated salary data. When she tested the model using the remaining 200,000 job listings, it accurately predicted the associated salaries 87 per cent of the time. 

In comparison, using only the job postings’ job titles and geographic locations yielded accurate predictions just 69 per cent of the time.

Ms Bana will attempt to characterise the contribution of various words to the salary prediction, in follow-up work. “Ideally, we will colour words within postings from red to green, where the darker red words are linked with lower salary and the darker green are linked with higher salary,” she said. 

Source: VentureBeat

(Links and quotes via original reporting)

The post-pandemic US job landscape is continuing to evolve, globalisation is continuing to push jobs to new locations and new technologies are transforming the nature of many occupations. In the face of such workplace turmoil, workers, employers and policymakers, could benefit from understanding which job characteristics lead to higher wages and mobility, VentureBeat reports.

Sarah Bana - a postdoctoral fellow at Stanford’s Digital Economy Lab, part of the Stanford Institute for Human-Centered Artificial Intelligence - is working towards exactly that. Ms Bana says a large dataset exists that might help to provide that understanding: the text of millions of online job postings. 

“Online data provides us with a tremendous opportunity to measure what matters,” she said.

Using artificial intelligence (AI) and machine learning, Ms Bana recently demonstrated that the words used in a dataset of more than one million online job postings explain 87 per cent of the variation in salaries across a vast proportion of the labour market. 

This is reportedly the first work to use such a large dataset of postings and to explore the relationship between postings and salaries. 

Ms Bana also experimented with injecting new text – adding a skill certificate, for example – into relevant job listings to see how these words changed the salary prediction.

“It turns out that we can use the text of job listings to evaluate the salary-relevant characteristics of jobs in close-to real time,” Ms Bana said. “This information could make applying for jobs more transparent and improve our approach to workforce education and training.”

An AI dataset of one million job postings 

In order to analyse how the text of online job postings relates to salaries, Ms Bana obtained more than one million pre-pandemic job postings from Greenwich.HR, which aggregates millions of job postings from online job board platforms. 

She then used BERT - one of the most advanced natural language processing (NLP) models available - to train an NLP model using the text of more than 800,000 of the job postings and their associated salary data. When she tested the model using the remaining 200,000 job listings, it accurately predicted the associated salaries 87 per cent of the time. 

In comparison, using only the job postings’ job titles and geographic locations yielded accurate predictions just 69 per cent of the time.

Ms Bana will attempt to characterise the contribution of various words to the salary prediction, in follow-up work. “Ideally, we will colour words within postings from red to green, where the darker red words are linked with lower salary and the darker green are linked with higher salary,” she said. 

Source: VentureBeat

(Links and quotes via original reporting)