Driver+ tried to put me in the wall today!

[SASSquatch]

Great post!

Unlike @thrill I do not have formal experience with the technical details behind machine learning, but I have some related experience and classes. I know to never underestimate technology, but I think safe and reliable self-driving cars are still a ways off. I hope it happens, it could just be a while.

My armchair thinking is that a human being driving a car applies not just "car-driving" knowledge while at the wheel, but also all the other life-experience of living in this world. Maybe that knowledge isn't tapped when the driving situation is "standard": well defined lanes, well-behaved traffic, no random non-vehicle objects or dynamics in the environment. But "standard" is in quotes because how much of the time is real driving in that kind of environment? Maybe mostly for some, very little for others.

Training a car to drive by collecting data about a road and other cars is necessary, but not sufficient. What about how kids chase after balls or ride bikes? What about how tree branches appear in different light and different seasons? Is that light through the mist and rain at dusk a porchlight or another car with a headlight out? It's going to take a long time for self-driving algorithms that just use data from driving to back fill all those other life experiences that we apply when driving. Again, my speculation.

Plus, the computational capacity of the human brain is . . . impressive. Getting a silicon equivalent in a car will take a while.

In any case, I am thankful for this thread, and related threads, as a heads up. My car is old enough that even the most basic driver assistance features didn't exist when it was made. I didn't plan on using anything from the Rivian in this regard, and now for sure I won't.

You hit the nail on the head. The difficulty with Machine Learning (ML) is that you typically need exorbitant amounts of data. ML models need robust data points of every possible scenario and they are very susceptible to noise from degraded data signals.

When you are selecting "I'm not a robot" when you do a CAPTCHA test for some sites you will be asked to pick out a "fill in the bank" object from a series of pictures. But the pictures typically are blurry or grainy images.

The human brain is exceptionally good at object recognition (even severely degraded images) and generalizing to other exemplars with very few data points. You don't need to see every possible example of an apple to distinguish it from an orange, but a ML model might.

All the things human vision is very good at - seeing and recognizing a shadow or a tree branch vs something that would cause serious damage if you struck it, ML and AI models struggle with. There is still a very wide gap here.

A great example comes from how are visual system works. The fovea, the center of the retina which has the highest visual acuity, is packed with photoreceptors called cones that provide color vision in light conditions. Rods line the outer or peripheral area of the retina and are used for low light/non color vision.

When you are walking around at night in low light conditions, the fovea is one giant scotoma or blindspot but you don't perceive a giant hole in your vision. Why? Because your brain is literally filling in everything in that blindspot with what it is seeing in the periphery and what it is expecting to see and it's damn good at it because if you hadn't just read what I wrote (or are really up to speed on the human vision literature) you would have never known your visual system was doing that.

That is next level amazing that AI and ML is nowhere close to replicating.

My personal bias is that we can augment the human with driving aids but that the human should always be in control of the system - until at least we remove humans from the equation all together. But that is for another thread...

Sponsored

 

[Airbuswillie]

I haven’t driven the truck yet. Should be getting it before the end of the year though. I have driven my wife’s Model 3 enough to know that I don’t trust ANY of the self driving features enough to really use them for more than just a few minutes here and there until something happens I don’t like and then they get turned off again and I drive like I have for the last 28 years of being a licensed driver. The idea behind them and some of the functionality is nice but the tech just isn’t there yet and the roads in the U.S. are just too varied and have construction and other terrible drivers and obstacle and all sorts of stuff and those cameras cannot possibly keep up. In my opinion, and that’s all it is, an opinion, you’re still safer on average to be driving yourself and maybe just use the cruise control for speed and that’s it. Like one other commenter said something about airline pilots have planes that can fly themselves, and while that’s true there are still a hundred things that can go wrong. I AM an airline pilot and the plane I fly can auto land in the worst visibility possible but there’s a list of about 30 things that have to go right and/or be functioning properly to even attempt an auto land. And those approaches and airport and so much more tightly controlled than the public road we all drive on. The technology is just not good enough and the roads aren’t good enough in enough places to really use it consistently. Keep driving yourself. It’s more fun anyway.

 

[Airbuswillie]

While I agree with your comments in principle, I will challenge you on the plane analogy. Modern aircraft have sophisticated autopilot features that not only can fly the plane when it reaches cruising altitude but can also take off and land in emergency situations. Pilots spend a significant amount of their flight time in autopilot because it has been shown to be safe and effective over millions of flight hours.

Airplanes aren't a great analogy as FAA requires 3 miles of separation and there are no "walls" in the sky to run into.

Personally, I would never fully trust autopilot in vehicles because the technology simply isn't there yet. Nearly all accidents occur because of operator (human error). The human brain is capable of 63 quintillion operations per second, which as of a ~ decade ago, constituted all known computing power worldwide. It's also the most efficient information processor ever conceived and can be powered by a ham sandwich and a glass of milk.

We've made strides since then in computer vision, neural networks, and power efficiency but it is light years away from replicating the human brain.

Keep your attention on the road, hands on the wheel, and keep those fingers away from your phone and you'll do fine.

I don’t know about the planes TAKING OFF on autopilot but besides that I couldn’t agree more with everything you said. Embraer or Honda Jet or Vision Jet I believe make a plane that in an emergency will find and tune into the nearest ILS and the autopilot will descend and make the approach and land without any human interaction I believe but that’s not common in other planes. That’s very new. And even then there are plenty of things that could go wrong.

 

[DJG]

Like many things, examples of when things go wrong get much more press/interest than when they go according to plan, especially when it's a machine doing something a human operator doesn't expect or desire. I've had the same behavior occur a couple of times in the R1T so far, and many before that in an Audi (both of which I deem to be highly competent at their intended purpose). However, for every one of those events, there have been many more where the system prevented an accident.

And that is the point. These systems are not expected to be perfect, but they are expected to be better on the whole than driving without them. And the evidence strongly suggests that they are. The human brain may be more capable at processing all the information of driving and reacting than these systems are, when the human is focused and engaged. But that's the thing with humans, we aren't always focused and engaged, no matter how much we clearly know we should be when operating a vehicle.

So, knowing this fact, there are two options. Expect all humans (not just yourself, ALL humans sharing the road with you) to remain 100% focused while driving and do away with these systems, because at best they are worse than us when attentive and focused and at worst they could cause an incident when they get bad data input. Or, recognize that the first option is not realistic, and therefore there needs to be redundancy built in to avoid negative events. Someone will look back at their screaming child, turn to talk to the passenger, fall asleep, drive drunk, day dream, etc. All the things we clearly know increase risk but yet we all do one or more of them frequently.

For any that doubt that any of these assistance systems are safe at all, much less safe than not having them, I'd ask this question - why is it that an insurer, who is highly regulated, would provide a discount on your policy if your vehicle is equipped with, and even more still, you use such a system? The answer is because despite single data points of close calls, etc. they have objective data that tells them that a policy owner using an assistance system is less likely to be in an accident and have a claim. To think that a company like Nationwide would discount policy holder premiums for using Driver+ on a brand new vehicle from a first time automaker without any empirical data on systems in general and more importantly Rivian's, is pretty naive if you ask me.

 

[kizamybute']

Like many things, examples of when things go wrong get much more press/interest than when they go according to plan, especially when it's a machine doing something a human operator doesn't expect or desire. I've had the same behavior occur a couple of times in the R1T so far, and many before that in an Audi (both of which I deem to be highly competent at their intended purpose). However, for every one of those events, there have been many more where the system prevented an accident.

And that is the point. These systems are not expected to be perfect, but they are expected to be better on the whole than driving without them. And the evidence strongly suggests that they are. The human brain may be more capable at processing all the information of driving and reacting than these systems are, when the human is focused and engaged. But that's the thing with humans, we aren't always focused and engaged, no matter how much we clearly know we should be when operating a vehicle.

So, knowing this fact, there are two options. Expect all humans (not just yourself, ALL humans sharing the road with you) to remain 100% focused while driving and do away with these systems, because at best they are worse than us when attentive and focused and at worst they could cause an incident when they get bad data input. Or, recognize that the first option is not realistic, and therefore there needs to be redundancy built in to avoid negative events. Someone will look back at their screaming child, turn to talk to the passenger, fall asleep, drive drunk, day dream, etc. All the things we clearly know increase risk but yet we all do one or more of them frequently.

For any that doubt that any of these assistance systems are safe at all, much less safe than not having them, I'd ask this question - why is it that an insurer, who is highly regulated, would provide a discount on your policy if your vehicle is equipped with, and even more still, you use such a system? The answer is because despite single data points of close calls, etc. they have objective data that tells them that a policy owner using an assistance system is less likely to be in an accident and have a claim. To think that a company like Nationwide would discount policy holder premiums for using Driver+ on a brand new vehicle from a first time automaker without any empirical data on systems in general and more importantly Rivian's, is pretty naive if you ask me.

I agree that they add safety to your drive WHEN USED PROPERLY. Which means, when you're paying very close attention. They can often be better for drivers that do get distracted, by keep those drivers in their lane, but those are the drivers that also may miss the dangerous scenario that could have been avoided had they been paying attention. So, some of it is catch 22. They can be beneficial when used as intended. They do make driving easier when they work properly. Just as drivers do, they make mistakes. Sometimes bad ones. The two, when properly combined together, should be much safer than just having a driver or just having an "auto-pilot" like system.

 

[DJG]

I agree that they add safety to your drive WHEN USED PROPERLY. Which means, when you're paying very close attention. They can often be better for drivers that do get distracted, by keep those drivers in their lane, but those are the drivers that also may miss the dangerous scenario that could have been avoided had they been paying attention. So, some of it is catch 22. They can be beneficial when used as intended. They do make driving easier when they work properly. Just as drivers do, they make mistakes. Sometimes bad ones. The two, when properly combined together, should be much safer than just having a driver or just having an "auto-pilot" like system.

It comes back to the difference of perspective between single events and the system on whole. I specifically use driver assistance systems when I know there will be times I am distracted (pretty much every drive), and so it is expected to decrease risk in those instances on average. It may run me into a wall on one occasion, but it will also prevent me running into a wall nine times for every one of those (anecdotal WAG from experience). That's the point of the system from a direct safety perspective. I would also say that over long periods of driving time, the lane centering reduces fatigue of the driver, so that it actually reduces the likelihood of needing to intervene at all, which also decreases risk.

That's the key takeaway. They aren't self driving systems and they aren't supposed to replace a human driving. They are built to reduce risk on a statistical large numbers basis, which they do very well. So it's illogical to have any discussion about their viability based on single events.

Here is the executive summary of DA systems from the NHTSA:
Overview
Driver assistance technologies hold the potential to reduce traffic crashes and save thousands of lives each year. In 2020, 38,824 people died in motor vehicle crashes — many of these crashes were tied to human error.

And a snippet from the conclusion:

We now know that driver assistance technologies are the right path toward safer roads. We will work diligently to bring you updated information whenever there are breakthroughs with new driver assistance technologies.


Starting this year, consumer vehicles in Europe are mandated to include ADAS features, including lane centering and lane keep assist.
The Commission expects that the proposed measures will help save over 25,000 lives and avoid at least 140,000 serious injuries by 2038. This will contribute to the EU's long-term goal of moving close to zero fatalities and serious injuries by 2050 ("Vision Zero").

 

[kizamybute']

It comes back to the difference of perspective between single events and the system on whole. I specifically use driver assistance systems when I know there will be times I am distracted (pretty much every drive), and so it is expected to decrease risk in those instances on average. It may run me into a wall on one occasion, but it will also prevent me running into a wall nine times for every one of those (anecdotal WAG from experience). That's the point of the system from a direct safety perspective. I would also say that over long periods of driving time, the lane centering reduces fatigue of the driver, so that it actually reduces the likelihood of needing to intervene at all, which also decreases risk.

That's the key takeaway. They aren't self driving systems and they aren't supposed to replace a human driving. They are built to reduce risk on a statistical large numbers basis, which they do very well. So it's illogical to have any discussion about their viability based on single events.

Here is the executive summary of DA systems from the NHTSA:
Overview
Driver assistance technologies hold the potential to reduce traffic crashes and save thousands of lives each year. In 2020, 38,824 people died in motor vehicle crashes — many of these crashes were tied to human error.

And a snippet from the conclusion:

We now know that driver assistance technologies are the right path toward safer roads. We will work diligently to bring you updated information whenever there are breakthroughs with new driver assistance technologies.


Starting this year, consumer vehicles in Europe are mandated to include ADAS features, including lane centering and lane keep assist.
The Commission expects that the proposed measures will help save over 25,000 lives and avoid at least 140,000 serious injuries by 2038. This will contribute to the EU's long-term goal of moving close to zero fatalities and serious injuries by 2050 ("Vision Zero").

I have found them to be GREAT in bumper to bumper traffic scenarios and wide open highway scenarios. Those are the times I use them most. High-speed traffic or mid-level traffic where many drivers are zig-zagging in and out of traffic, or you go from 65 mph to 0 mph on a regular basis, I generally take over in those scenarios.

I get your point. But, knock on wood, for me, I've driven for decades without ever accidentally running into a wall. Between the Tesla having tried to do it several times, and now two relatively close calls with the Rivian, had I chosen to allow myself to be distracted for either of those very short periods of time, the results would have been bad, especially the ones where it would have hit the end of the wall, rather than just side-slapped it. Your analogy of 9 out of 10 times its better, is actually pretty horrible odds. We need to be 999,999 out of 1,000,000 times it's better. That's when we'll have systems where even if they are still only LEVEL 2 systems, you might be able to feel comfortable with allowing yourself to get distracted while driving. I certainly don't want my car/truck to wreck itself every 1 out of every 10 times!!

But yes, they do offer a lot of added safety, but again, when used properly. Especially just traffic aware cruise control. I'm sure its saved countless of rear-end accidents from happening.

 

[DJG]

Your analogy of 9 out of 10 times its better, is actually pretty horrible odds. We need to be 999,999 out of 1,000,000 times it's better. That's when we'll have systems where even if they are still only LEVEL 2 systems, you might be able to feel comfortable with allowing yourself to get distracted while driving. I certainly don't want my car/truck to wreck itself every 1 out of every 10 times!!

Actually, my point is that they only need to avoid risk 1.01 times for every 1 time they create it for them to be adding value and reducing systemic risk in vehicle transportation. Doesn't mean the work is over, because the goal is as close to zero risk as possible. The EU actually has a real goal of zero traffic fatalities by 2050, which will only be achieved by continued advancement in these systems. Human beings alone reached the peak of their abilities hundreds of years ago. And ironically, the only/best way they advance is for people to use them now. They are far from perfect, but you are relatively safer with them on than with them off.

 

[Scoiatael]

Having had a Tesla with "autopilot" and now the Rivian, I never use these systems. They are not ready for primetime. The only one I'll both with is the distance controlling cruise control.

I used autopilot quite a lot on my Model Y. Never had any issues with it. Over 1.5 years it phantom braked 3 times. It also did a good job swerving out of the way when a big rig started to creep into my lane. I learned to trust it after a few months.

Driver+ so far I really don't trust it except at night. Using it at night has been near flawless for me. But during the day I've had a few phantom brakes, and some questionable lane centering. Yesterday I tried Driver+ on a freeway I haven't taken the R1T on yet before. Nothing special, a few curves but nothing too sharp. All the lanes are clearly marked, pavement is all the same color. Driver+ on a small curve tried to start pushing me into the right lane. I've been on sharper curves before that Driver+ handled fine. No reason for it to try to push me into the right lane.

All the routes that Driver+ is failing at so far have been perfect for Tesla's autopilot.

 

[kizamybute']

They are far from perfect, but you are relatively safer with them on than with them off.

Absolutely agree, if you combine the driver with the auto driver assist capabilities together, it's absolutely 100%, definitely safer than without. The only time it becomes an issue is when the drivers allow themselves to get distracted because they have the auto driver assist features. Under those circumstances, then it could become more dangerous than just having the driver.

These systems are incredible. Elon Musk originally thought we'd have self driving cars 3 years ago. They did such an incredible job figuring out the first 95% of the equation, but then realized that, that last 5% is really difficult to achieve. To no fault of their own, just that that last 5% is where the human brain can analyze a unique, one of kind situation in an instant and react. Very difficult to program a computer to be able to handle any and all potential situations as we ourselves don't even know what many of those will be.

Personally, I think the only way we get to Level 4 or 5 full self driving is by having all cars capable and communicating with each other. Simply trying to analyze and calculate everything via cameras, lidar, radar or whatever else they come up with, will always have some aspects that haven't been accounted for. Technically, a Tesla right now can drive itself. It's very jerky, very slow in doing so, misses a lot of turns, attempts to turn left in front of oncoming traffic, doesn't always see pedestrians, makes right turns into the path of oncoming traffic that were making left turns into the same lanes. If it's not presented with any unique scenarios, it could possibly make it from point A to point B completely on its own. Unfortunately, most of the time, something will come up that it struggles with and can't figure out.

I like that Elon released the capability, not only because it gives the owners a chance to experience it, but is one of the best ways for it to collect real world data and hopefully learn from its mistakes via driver input and corrections.

I don't see us being able to release a vehicle to go from L.A. to New York as Elon had hoped to do 2-3 years ago, anytime soon. We're still years away. Tesla certainly has the most advanced system that has been released to customers. GM's system, in terms of highway use, is probably the best, better than Tesla on the highway. Ford is getting there. Rivian, still has quite a ways to go. Having to disable every time I go through what it deems to be a tunnel (freeway overpass) or many other occasions does sometimes make it more work to pay attention to what it wants me to do than just driving myself. But, I love tech and love that we get to see it grow.

My bet is that GM is the first to release a fully usable Level 3 system. They don't get as bold and release things in "beta", but I think they are close. Probably closer than Tesla or any others. I do hope that Rivian's system at least gets to Tesla's basic AP level soon. Lane keep assist and traffic aware cruise control are now both pretty basic functions that it seems most have figured out. The Rivian still only allows the lane keep assist (not counting the lane correction system where it jerks you back into your lane if you drift out of it without anything engaged), to be engaged in mapped locations. And again, makes you disable it too often for bridges, road construction zones, etc, whereas most others continue keeping you centered in the lane simply based on lane markings.

 

[iansriv]

 

[kizamybute']

Yep, these are the bad eggs that ruin it for everyone. And of course, the ones the media focuses on.

When AP first came out on the Tesla, I was able to go 150 miles on the 40 freeway without a single "nag" to move the steering wheel. Was AMAZING. Then, all the idiots started posting videos of themselves in the back seat or sleeping or whatever and now the nags are more of a hassle than they're worth. The Rivian I will say is much better than the Tesla in detecting my hands on the wheel. It never nags me. The Tesla's. all struggled to feel my hand and constantly got the warning to wiggle the wheel so it knew I was there. Ended up turning it off most of the time because it just got more tiresome having to wiggle the wheel every 13 seconds than to just put my hand on the wheel and hold it straight without AP engaged.

 

[DJG]

Personally, I think the only way we get to Level 4 or 5 full self driving is by having all cars capable and communicating with each other. Simply trying to analyze and calculate everything via cameras, lidar, radar or whatever else they come up with, will always have some aspects that haven't been accounted for.

That's correct, and I think that's exactly what will happen. There will come a time when a city/cities prohibit human controlled vehicles from operating within it's CBD. It will have the following benefits:

1. By requiring all vehicles to be operating under the same "system", they are all connected and therefore accidents with each other can be 100% avoided. All accidents can likely be avoided, because they'd also be connected to the transportation grid for things like traffic lights and crosswalks.
2. The need for parking in dense urban settings goes down dramatically, perhaps close to zero. The only need would be in residential buildings, but because of the prohibition on driving, you'd like take a robotaxi to a garage outside the downtown to get your car.
3. Studies have shown that a significant percentage (upwards of 40-50%) of traffic in urban settings is from vehicles looking for a parking space. That all goes away, so the number of vehicles on the streets is nearly cut in half.
4. Removing the need for on street parking opens up opportunities for outside dining, parks, or whatever uses can be come up with to recapture the sidewalks. There will be a lot more pedestrian malls like Pearl St. in Boulder.

By the way, before anyone jumps on it, I haven't mentioned the big T in any of this. I don't care where the vehicles come from, I just think this is where we are headed. Cities like London have long already had restrictions on driving in the urban core.

 

[Nine_One_Six_R1S]

My bet is that GM is the first to release a fully usable Level 3 system.

IIRC, I read a few days ago that Mercedes Benz has been the first Car maker to release a L3 System. It's currently available only on S Class and EQS. Can't recall if it's available in the US

 

[SASSquatch]

I don’t know about the planes TAKING OFF on autopilot but besides that I couldn’t agree more with everything you said. Embraer or Honda Jet or Vision Jet I believe make a plane that in an emergency will find and tune into the nearest ILS and the autopilot will descend and make the approach and land without any human interaction I believe but that’s not common in other planes. That’s very new. And even then there are plenty of things that could go wrong.

https://www.airbus.com/en/newsroom/...s-first-fully-automatic-vision-based-take-off

Sponsored