The jobs we'll lose to machines

Machine learning isn't just for simple tasks like assessing credit risk and sorting mail anymore -- today, it's capable of far more complex applications, like grading essays and diagnosing diseases. With these advances comes an uneasy question: Will a robot do your job in the future?Published: Mon Aug 08 2016

So this is my niece. Her name is Yahli. She is nine months old. Her mum is a doctor, and her dad is a lawyer. By the time Yahli goes to college, the jobs her parents do are going to look dramatically different.

In 2013, researchers at Oxford University did a study on the future of work. They concluded that almost one in every two jobs have a high risk of being automated by machines. Machine learning is the technology that's responsible for most of this disruption. It's the most powerful branch of artificial intelligence. It allows machines to learn from data and mimic some of the things that humans can do. My company, Kaggle, operates on the cutting edge of machine learning. We bring together hundreds of thousands of experts to solve important problems for industry and academia. This gives us a unique perspective on what machines can do, what they can't do and what jobs they might automate or threaten.

Machine learning started making its way into industry in the early '90s. It started with relatively simple tasks. It started with things like assessing credit risk from loan applications, sorting the mail by reading handwritten characters from zip codes. Over the past few years, we have made dramatic breakthroughs. Machine learning is now capable of far, far more complex tasks. In 2012, Kaggle challenged its community to build an algorithm that could grade high-school essays. The winning algorithms were able to match the grades given by human teachers. Last year, we issued an even more difficult challenge. Can you take images of the eye and diagnose an eye disease called diabetic retinopathy? Again, the winning algorithms were able to match the diagnoses given by human ophthalmologists.

Now, given the right data, machines are going to outperform humans at tasks like this. A teacher might read 10,000 essays over a 40-year career. An ophthalmologist might see 50,000 eyes. A machine can read millions of essays or see millions of eyes within minutes. We have no chance of competing against machines on frequent, high-volume tasks.

But there are things we can do that machines can't do. Where machines have made very little progress is in tackling novel situations. They can't handle things they haven't seen many times before. The fundamental limitations of machine learning is that it needs to learn from large volumes of past data. Now, humans don't. We have the ability to connect seemingly disparate threads to solve problems we've never seen before.

Percy Spencer was a physicist working on radar during World War II, when he noticed the magnetron was melting his chocolate bar. He was able to connect his understanding of electromagnetic radiation with his knowledge of cooking in order to invent -- any guesses? -- the microwave oven.

Now, this is a particularly remarkable example of creativity. But this sort of cross-pollination happens for each of us in small ways thousands of times per day. Machines cannot compete with us when it comes to tackling novel situations, and this puts a fundamental limit on the human tasks that machines will automate.

So what does this mean for the future of work? The future state of any single job lies in the answer to a single question: To what extent is that job reducible to frequent, high-volume tasks, and to what extent does it involve tackling novel situations? On frequent, high-volume tasks, machines are getting smarter and smarter. Today they grade essays. They diagnose certain diseases. Over coming years, they're going to conduct our audits, and they're going to read boilerplate from legal contracts. Accountants and lawyers are still needed. They're going to be needed for complex tax structuring, for pathbreaking litigation. But machines will shrink their ranks and make these jobs harder to come by.

Now, as mentioned, machines are not making progress on novel situations. The copy behind a marketing campaign needs to grab consumers' attention. It has to stand out from the crowd. Business strategy means finding gaps in the market, things that nobody else is doing. It will be humans that are creating the copy behind our marketing campaigns, and it will be humans that are developing our business strategy.

So Yahli, whatever you decide to do, let every day bring you a new challenge. If it does, then you will stay ahead of the machines.

Thank you.