It seems obvious to a human, but to a computer it's actually a surprisingly hard problem!
For my car to understand that another car with its turn signal on may be about to move into my lane, it had to know a ton of things:
- What a turn signal looks like, not just on this specific car but on any vehicle it might encounter.
- The difference between a turn signal and a brake light.
- The difference between an active and inactive turn signal.
- The difference between a right and left turn signal.
- The difference between my lane and other lanes.
- How to recognize all of these things at daytime, at nighttime, in rain, in fog, in snow, etc.
- What it means when a car in the lane to my left uses its right turn signal.
- The speed of the other car relative to my car.
...and so on.
All of this seems very intuitive to a human, but to a computer it's the culmination of years and years of painstaking neural network training with terabytes and terabytes worth of training data.
Until very recently, Tesla Autopilot primarily used forward-looking radar and very basic vision for determining where other cars are and what they're doing (vision was mostly used for spotting lane lines). This meant that the car was mostly only aware of other cars in terms of their being objects in space moving at various speeds. It couldn't really tell when one of those objects intended to do something; only when it had done something. So if another car put on its blinker, my car wouldn't actually do anything until that car actually moved into my lane or impeded my path in some way.
But recent Autopilot updates have made it a lot smarter, and it's now using eight cameras to see all the cars around me. It still doesn't have much in the way of intuition — it still mostly reacts rather than predicting — but this incident was one of the first strong indications I've seen that the car really was predicting what was about to happen based primarily on visual information.
It's a small but significant step on the path to much bigger things. 🙂