2017 Artificial Intelligence report card

Average Human IQ: 100
Machine IQ: 7
Cooperative IQ: 120

140 character summary: AI is still dumb, but it can be helpful.

This year has been surreal in a lot of ways, but my favorite is this: non-ironic conversations In the cultural mainstream about super intelligent AI subjugating humanity. Experts and pundits were split. Some pooh-poohed the possibility and some raised the alarm. Opinions on the topic are widely varied and strongly held, but wherever you fall, it bears asking the question, Just how smart is AI today? If the machines were to rise right now, would we have a chance?

If it were a contest of multiplication, the answer is no. When I was in the third grade, my intelligence was measured by how long it took me to do a hundred multiplication problems. But even the earliest computers were much better at multiplication than any human. In fact, a 2017 laptop can do multiplication problems faster than all the humans in the world combined. On the other hand, if it came down to the ability to assemble Swedish furniture based on pictorial instructions, humans would definitely come out on top, particularly if there were a few missing parts.

The fact that machines are better at something than humans is not new. Recently, there have been a few notable victories added to the machines’ tally, including safe driving and detecting lung cancer and skin cancer. 2017 has added a couple more:

Machine IQ

So just how close are machines to becoming smarter than humans? The most common measure of human intelligence is the intelligence quotient or IQ. To determine your IQ, you have to take a bunch of tests, each measuring a different aspect of your cognitive function. When you are done, you are compared to a large group of other people who have taken the same tests. If your scores are average, you get an IQ score of 100. If you do better than average, your IQ is higher, and if you do worse than average it’s lower.

We can modify this measure a bit to create a Machine IQ. If machine does something as well as an average human, then it would have a Machine IQ of 100. If it does something twice as well as a human, we would give it a Machine IQ of 2 x 100 = 200, and if it only does 1/10th as well as a human, it would earn a Machine IQ of 1/10 x 100 = 10. So, if a human is a thousand times better at the Swedish furniture test, based on that we might conclude that Machine IQ tops out at 0.1. However, if we base it on multiplication, Machine IQ is 1 trillion.

To get a well-rounded estimate of Machine IQ It is necessary to consider AI's capability across a range of tasks. Here’s the method I used. With a pint of strong IPA in a comfortable corner, I considered typical human activities over the course of year. Which of them could an AI perform today, without requiring any new technological advances? How quickly could it learn them? How long wout it take to complete the task? How good would the AI be at it? Better than me? By how much?

For all its successes, there are a lot of things I do that AI hasn't touched yet. Determining a dog's breed from an image? [Better than me.] Petting a dog? [Not even attempted.] Planning shortest traffic routes? [Better than me.] Predicting whether an Audi driver will slow down if I step into the crosswalk? [Out of its reach.] Finding errors in my code? [Better than me.] Learning how to use an under-documented machine learning library with breaking changes in every version? [Don't make me laugh.]

After completing this non-rigorous survey of activities, I concluded that on balance AI can do about 7% of what a typical human do, or do what a human does about 7% as well. That’s a Machine IQ of 7.

If that seems low to you, consider a few facts that are easy for us to forget in our excitement about AI’s achievements:

For all of our inefficiencies and irrationality (or perhaps thanks to them), humans excel at adaptation. Our facility in adapting is so great that we are unconsciously competent, handling a torrent of minor deviations and surprises each waking moment. As a species we may resist big changes, but when we rise to them, we can adapt to nearly anything. This includes the loss of a limb, our vision, and half a cerebral cortex. Jobs least likely to be done by a machine are the ones that require the most variety and the most adaptation. Machines are approaching the point where they can perform almost any single skill better than a human, but they can’t touch a human when comes to variety, and they’re not even in the same league when it comes to adaptation.

Cooperative IQ

The tremendous asymmetry in human and machine performance shows what we knew all along: humans and machines are good at different things. Yes, my phone is better at multiplication than I am, but I don’t actually spend much time doing multiplication. In fact, whenever I need to calculate a tip on a bar tab, I make my phone do my multiplication for me. My phone does what it’s better at, so that I can do what I’m better at - nodding a goodbye to the bartender and anticipating how my driver is going to misinterpret the address I entered.

Viewed from this direction, it also makes sense to ask another question: How much smarter are we with AI than without? How much more can I do, thanks to algorithms? We can put a number of this, called the Cooperative IQ. If AI doesn’t help us achieve anything more than we could do on our own, it multiplies our output by 100%, no change. If it doubles our output, that would be a Cooperative IQ of 200%. And if it hinders us such that we can only achieve half as much, we would assign it a Cooperative IQ of 50% .

To get an overall Cooperative IQ score, I repeated the introspection exercise and concluded that AI makes us 20% more effective, giving it a Cooperative IQ of 120. Here are some of the cases that factored in:

AI still has a very long way to go before it is smarter than humans, but it is quite useful as a partner. A hammer does not need to be very smart to greatly improve our ability to drive nails. AI is not yet a serious contender as a competitive intelligence but is becoming valuable as a cooperative one.