Tesla ‘Full Self-Driving’ tried to drive owner into a lake, viral video shows

[Time2Roll]

It's largely irrelevant because regardless of the destination and IP, the car was in FSD, driving on a residential street, signaled a right turn and turned onto a boat ramp when it could have turned into a driveway, if it was seeking to turn around, and if it was trying to turn around, it could have stopped in a car length, then backed out, but it accelerated forward after the turn.

Show me the map, route and destination on the map. Good chance as any the Tesla was directed to coordinates at the end of the launch ramp.

Sponsored

 

[mkhuffman]

Show me the map, route and destination on the map. Good chance as any the Tesla was directed to coordinates at the end of the launch ramp.

Actually it is highly likely they purposely routed the car onto the ramp just to see what it would do. Any proof they didn't do that?

 

[Time2Roll]

Actually it is highly likely they purposely routed the car onto the ramp just to see what it would do. Any proof they didn't do that?

Exactly.

 

[skyguyscott]

Show me the map, route and destination on the map. Good chance as any the Tesla was directed to coordinates at the end of the launch ramp.

Such a possibility becomes increasingly unlikely when you examine the evidence on the video where the Tesla is accelerating after the turn and does not attempt to slow down mere feet from the water. If the input coordinates were at the end of the ramp, why then does the car accelerate rather than slow down? Why, if the coordinates were beyond the water line, does the visual system fail to recognize and respond to the hazard?

I suppose a possible scenario to explain this would be if the driver intentionally put in coordinates in the center of the lake, but then, does or does not the Tesla navigation system know where existing bodies of water are? Another possibility, if you want to go full conspiracy, is that the driver employed some sort of GPS spoofing device to confuse the Tesla, or that they hacked the Tesla and made it malfunction on purpose, or that the entire video is an elaborate hoax produced by media professionals or AI.

Or you could follow Occam's razor and deduce the most likely scenario is that a visual system with known and documented limitations failed to negotiate the proper route, then failed to recognize and respond appropriately to a hazard, behavior it is known to have exhibited before. While nefarious actors can't be entirely ruled out short of an intense investigation, we also should not be so quick to rule out the possibility that FSD has real issues.

BTW, does or does not Tesla get telemetry from its vehicles and is able to determine what the programed destination was, along with other vehicle parameters. If so, I suspect that Tesla will be eager to report its findings, especially if this was a set up.

 

[Time2Roll]

Or just show the map in the video. Seems like something is hidden.

afaik the only reason to go down the ramp would be the operator directed it by choice.
Or is someone suggesting there is an old road across the lake seasonally when it drys up in summer.
Just does not add up.

 

[skyguyscott]

Or just show the map in the video. Seems like something is hidden.

afaik the only reason to go down the ramp would be the operator directed it by choice.
Or is someone suggesting there is an old road across the lake seasonally when it drys up in summer.
Just does not add up.

FSD is software controlling hardware, it does not reason, it calculates, it receives input from sensors and executes code. I'm not sure even those working at the highest levels in AI would characterize their algorithms as reasoning in the human sense. Logical, yes, able to draw inferences, sure, as smart as a 15-year-old driving with a leaner's permit? Not yet.

 

[captainjp]

it receives input from sensors and executes code.

Yes, and the original point is that FSD is limited in this area. The insistence that additional sensor input (radar, lidar) would not improve FSD performance and prevent edge cases like this is foolish and ignorant. FSD will plateau at some point if continued with camera-only systems. It’s inevitable that it will eventually be surpassed by other systems. I personally would not feel very safe getting in a robotaxi knowing this.
I will say that FSD is impressive even with this limitation. However, that is mainly due to the many miles of ground truth data collected from the millions of Tesla vehicles on the road.

 

[mkhuffman]

Such a possibility becomes increasingly unlikely when you examine the evidence on the video where the Tesla is accelerating after the turn and does not attempt to slow down mere feet from the water. If the input coordinates were at the end of the ramp, why then does the car accelerate rather than slow down? Why, if the coordinates were beyond the water line, does the visual system fail to recognize and respond to the hazard?

I suppose a possible scenario to explain this would be if the driver intentionally put in coordinates in the center of the lake, but then, does or does not the Tesla navigation system know where existing bodies of water are? Another possibility, if you want to go full conspiracy, is that the driver employed some sort of GPS spoofing device to confuse the Tesla, or that they hacked the Tesla and made it malfunction on purpose, or that the entire video is an elaborate hoax produced by media professionals or AI.

Or you could follow Occam's razor and deduce the most likely scenario is that a visual system with known and documented limitations failed to negotiate the proper route, then failed to recognize and respond appropriately to a hazard, behavior it is known to have exhibited before. While nefarious actors can't be entirely ruled out short of an intense investigation, we also should not be so quick to rule out the possibility that FSD has real issues.

BTW, does or does not Tesla get telemetry from its vehicles and is able to determine what the programed destination was, along with other vehicle parameters. If so, I suspect that Tesla will be eager to report its findings, especially if this was a set up.

What was the destination of the route?

 

[captainjp]

What was the destination of the route?

Pretty sure that’s irrelevant. Regardless of destination, the vehicle should not have taken that route.
According to your logic, if I put a destination in the middle of the Atlantic Ocean, the car will attempt to navigate to it?

 

[mkhuffman]

Pretty sure that’s irrelevant. Regardless of destination, the vehicle should not have taken that route.
According to your logic, if I put a destination in the middle of the Atlantic Ocean, the car will attempt to navigate to it?

Yes, it will. Because you told it to. In this case, the driver entered a route that took the car down the ramp. What was the destination? That is my simple question, and it definitely does matter. We don't have enough information to make a full judgment until we know that. And it is hidden from us. Why?

IMO the camera should have seen the lake and stopped the car. With my imperfect eyes I can clearly see the lake, so the car made a mistake by not stopping. But...

What if a road is flooded and you route the car through the water? What do you expect the car to do? Does the car know how deep the water is? Should it stop even if the water is two inches deep? What about two feet?

Does the car trust the driver's judgements regarding the flooding and keep going, or does it stop and force manual driving? There might be something like this happening in the lake scenario. We don't know because they have not provided enough information to know. Which is very suspicious.

 

[captainjp]

Yes, it will. Because you told it to.

I’m almost certain that this is completely false and absurd. There’s no way in hell that the system would allow that. In any case if this were true, you find that to be acceptable behavior for such an advanced system?

 

[Dark-Fx]

Yes, it will. Because you told it to. In this case, the driver entered a route that took the car down the ramp. What was the destination? That is my simple question, and it definitely does matter. We don't have enough information to make a full judgment until we know that. And it is hidden from us. Why?

IMO the camera should have seen the lake and stopped the car. With my imperfect eyes I can clearly see the lake, so the car made a mistake by not stopping. But...

What if a road is flooded and you route the car through the water? What do you expect the car to do? Does the car know how deep the water is? Should it stop even if the water is two inches deep? What about two feet?

Does the car trust the driver's judgements regarding the flooding and keep going, or does it stop and force manual driving? There might be something like this happening in the lake scenario. We don't know because they have not provided enough information to know. Which is very suspicious.

If I had a Tesla with FSD, I'd be going to it right now to make it navigate to a point in the middle of a nearby lake and seeing what it suggests for a route.

You should almost never drive through an unfamiliar road that has water across it for an extended distance. Every road is an unfamiliar road to FSD, so one should expect it to not attempt it, especially if it's not following another vehicle. It should stop and re-route.

 

[mkhuffman]

If I had a Tesla with FSD, I'd be going to it right now to make it navigate to a point in the middle of a nearby lake and seeing what it suggests for a route.

If I had one, I would do the same thing. If there is a boat ramp into the lake, even better.

Will FSD drive off road, or only on roads? If you map it to drive through a field, like sometimes happens at event parking, will it do it, or refuse? Should it refuse? What about if you live in a rural area with a driveway that isn't really a driveway? Should it stop and refuse to drive on that?

If I had a FSD Tesla, I would definitely test all those scenarios.

I’m almost certain that this is completely false and absurd. There’s no way in hell that the system would allow that. In any case if this were true, you find that to be acceptable behavior for such an advanced system?

If it only drives on roads, then it cannot replace a human driver, right?

Even with LiDAR, the truck will need to decide if it should drive the route you are forcing it to take. If you want to go down a dirt road, and it really does not look like a dirt road, it should refuse. Or should it? If you want it to drive through that water on the road, it should refuse. Or should it?

 

[SwampNut]

About once a year I drive on a road that spends half the year under water. I wonder what FSD would do. I think I can borrow an X to try it out. It’s probably submerged right now.

 

[captainjp]

If I had one, I would do the same thing. If there is a boat ramp into the lake, even better.

Will FSD drive off road, or only on roads? If you map it to drive through a field, like sometimes happens at event parking, will it do it, or refuse? Should it refuse? What about if you live in a rural area with a driveway that isn't really a driveway? Should it stop and refuse to drive on that?

If I had a FSD Tesla, I would definitely test all those scenarios.



If it only drives on roads, then it cannot replace a human driver, right?

Even with LiDAR, the truck will need to decide if it should drive the route you are forcing it to take. If you want to go down a dirt road, and it really does not look like a dirt road, it should refuse. Or should it? If you want it to drive through that water on the road, it should refuse. Or should it?

This is where map data and geofencing comes into play. You make a good point with your uncertainty. It shows that we’re just not there yet. In these scenarios, a prudent driver would stop and assess. Take all of the variables into account. Should I proceed or shouldn’t I? Maybe get out of the vehicle and look around. Obviously the vehicle is not capable of this type of logic. At the very least, at times of uncertainty, it should prompt the driver and maybe ask for permission to proceed.
This is why Waymo is far more trustworthy. Geo fencing in conjunction with the sensor suite provides multiple levels of redundancy.

Sponsored