mpshizzle
Well-Known Member
- Thread starter
- #1
I recommend watching the video on my YouTube Channel Thunder Volt Auto:
There's a lot more context and helpful visuals.
While I was at Rivian's Autonomy day, I was able to go on a demo drive of Rivian's Point to Point self driving system (Their version of FSD). You all have probably seen everyone elses coverage of these demo drives. All of us did the same route in the same set of cars. So rather than add to the pile of already great overviews out there, we're going to do this the mpshizzle way. We're going to "pixel peep". We're going to tear into the nuances of the software that nobody else is really talking about. I rode along, kept my eyes peeled, and found some fascinating details about how this demo was actually working.
Here is what I found: (Disclaimer, none of this has been explicitly confirmed by Rivian, these are just my observations)
First, a quick recap of my specific demo loop. It was incredibly smooth and controlled. The R1S drove like a very passive, safe driver.
We only had two disengagements
1. Peer Pressure: Two cars in front of us ran a red light. The Rivian saw them go and thought, "Oh, I guess we’re doing this now?" and tried to follow. The safety driver had to stop it so we didn't become the third jerk running the light.
2. Lane Confusion: We approached a line of cars moving right for a turn lane. The system wasn't quite sure if it was allowed to merge over yet, so the human helper stepped in.
Aside from that? Drama-free. But let’s look under the hood at the weird software stuff. These are observations I made about the prototype software package made for this event.
PRE PRODUCTION SOFTWARE ODDITIES
Right off the bat, I noticed the system was silent. No engagement or disengagement chimes.
Why? Well, maybe the engineers in Palo Alto just got sick of hearing chimes all day during testing. But more likely, for this press event, they didn't want the system calling attention to itself every time it needed help. To be fair, the safety drivers were verbally calling out disengagements, so they weren't trying to hide anything—they just muted the notifications.
Another oddity I saw had to do with co-steer. In the current production software (Highway Assist), if you tug the steering wheel a little bit, you can nudge it one way or another. If you tug too hard, it disengages. In this engineering build, it appears "Co-Steer" let you tug the wheel as much as you wanted, and the system stayed on. At one point, when the driver took over for that second disengagement, she pulled quite far on the wheel, and she had to manually hit the stalk to disengage it. That's something that's just a setting for the demo, that wouldn't make it into a production build, but interesting nonetheless.
I also noticed something odd with the nav. Not from my drive, but from watching Bearded Tesla Guy's drive. The center screen showed Google Maps, but I’m pretty sure the R1 was ignoring it. It seemed like the route was hardcoded into the backend for the demo loop. At one point on his drive, Google Maps suggested a weird maneuver involving a right turn followed by an immediate U-turn. The Rivian ignored that nonsense and just made the logical left turn. Additionally, it began driving the route before the driver hit "go" on the nav. Honestly though, this makes sense. Why go to all the effort to build hooks into the Google nav API connection for a 2 day demo? Just hardcode it and drive. When this software becomes production ready, of course then they will make that connection with the nav.
DRIVING MODEL OBSERVATIONS
This was my favorite thing I learned from the engineers. Apparently, early versions of this software drove FAST. Why? Because the AI is trained on customer data. And frankly, many Rivian owners have heavy feet (yes, me too). The system learned from people driving high-performance EVs and assumed, "Okay, we whip around now". They actually had to filter the training data to find more moderate, calm driving scenarios to teach the model to settle down. And honestly I never would have known any different on our drive. It was very calm.
Another interesting training tidbit: we usually think about training AI by showing it "good" driving. But Rivian is also using negative reinforcement. They feed the model scenarios where human drivers messed up—specifically times when Automatic Emergency Braking (AEB) had to kick in because someone was texting or distracted. By telling the AI, "See this? Don't do this," the model actually became a safer driver.
They also taught it to handle speed bumps not by writing code that says `IF speed_bump THEN slow_down`, but by feeding it examples of humans slowing down for bumps. The system figured out the logic on its own.
The driving model isn't perfect yet, though. It struggled a bit with merging and didn't seem to fully grasp pedestrian behavior just yet.
It also seems they're still figuring out how to adequately put reigns on the AI system. They had it set up to not do right on red for this demo. Though apparently, in some other demo drives (not mine), it decided to make the turn anyway (AI has a mind of its own sometimes). I'm assuming "right on red" won't be a prohibited behavior on production versions. My guess is this was more of them just demoing the guard rails they CAN put on the AI (though they didn't ENTIRELY work), along with being cautions until they're sure the vehicle can adequately guage cross traffic.
It also seems that Rivian is getting more targeted with the data they collect. Instead of just grabbing large swaths of driving (needed early on for basic driving dynamics), they are now targeting more specific gaps in the model's behavior. The big one coming up? Parking. Currently, privacy policies stop the car from recording the first and last three minutes of a drive—which is exactly when you park. They are updating this to record those segments unless you are at home or work, so the fleet can finally learn how to navigate a parking lot.
Conclusion
If you follow the competition, this is like "FSD 12". We are basically looking at Rivian’s version of an end-to-end neural net with no hard-coded rules for learned behavior.
It’s not production-ready yet. It needs to get better at things like merges and pedestrians. But the Gen 2 vehicles have the compute power to handle it, and the foundation is solid. I can't wait to see where it goes next.
If you found this information interesting, useful, or just entertaining and you're buying a Rivian, use my referral code to get up to 500 points in the Gear Shop (and free charging!): https://rivian.com/configurations/list?reprCode=MIKE4580282
There's a lot more context and helpful visuals.
While I was at Rivian's Autonomy day, I was able to go on a demo drive of Rivian's Point to Point self driving system (Their version of FSD). You all have probably seen everyone elses coverage of these demo drives. All of us did the same route in the same set of cars. So rather than add to the pile of already great overviews out there, we're going to do this the mpshizzle way. We're going to "pixel peep". We're going to tear into the nuances of the software that nobody else is really talking about. I rode along, kept my eyes peeled, and found some fascinating details about how this demo was actually working.
Here is what I found: (Disclaimer, none of this has been explicitly confirmed by Rivian, these are just my observations)
First, a quick recap of my specific demo loop. It was incredibly smooth and controlled. The R1S drove like a very passive, safe driver.
We only had two disengagements
1. Peer Pressure: Two cars in front of us ran a red light. The Rivian saw them go and thought, "Oh, I guess we’re doing this now?" and tried to follow. The safety driver had to stop it so we didn't become the third jerk running the light.
2. Lane Confusion: We approached a line of cars moving right for a turn lane. The system wasn't quite sure if it was allowed to merge over yet, so the human helper stepped in.
Aside from that? Drama-free. But let’s look under the hood at the weird software stuff. These are observations I made about the prototype software package made for this event.
PRE PRODUCTION SOFTWARE ODDITIES
Right off the bat, I noticed the system was silent. No engagement or disengagement chimes.
Why? Well, maybe the engineers in Palo Alto just got sick of hearing chimes all day during testing. But more likely, for this press event, they didn't want the system calling attention to itself every time it needed help. To be fair, the safety drivers were verbally calling out disengagements, so they weren't trying to hide anything—they just muted the notifications.
Another oddity I saw had to do with co-steer. In the current production software (Highway Assist), if you tug the steering wheel a little bit, you can nudge it one way or another. If you tug too hard, it disengages. In this engineering build, it appears "Co-Steer" let you tug the wheel as much as you wanted, and the system stayed on. At one point, when the driver took over for that second disengagement, she pulled quite far on the wheel, and she had to manually hit the stalk to disengage it. That's something that's just a setting for the demo, that wouldn't make it into a production build, but interesting nonetheless.
I also noticed something odd with the nav. Not from my drive, but from watching Bearded Tesla Guy's drive. The center screen showed Google Maps, but I’m pretty sure the R1 was ignoring it. It seemed like the route was hardcoded into the backend for the demo loop. At one point on his drive, Google Maps suggested a weird maneuver involving a right turn followed by an immediate U-turn. The Rivian ignored that nonsense and just made the logical left turn. Additionally, it began driving the route before the driver hit "go" on the nav. Honestly though, this makes sense. Why go to all the effort to build hooks into the Google nav API connection for a 2 day demo? Just hardcode it and drive. When this software becomes production ready, of course then they will make that connection with the nav.
DRIVING MODEL OBSERVATIONS
This was my favorite thing I learned from the engineers. Apparently, early versions of this software drove FAST. Why? Because the AI is trained on customer data. And frankly, many Rivian owners have heavy feet (yes, me too). The system learned from people driving high-performance EVs and assumed, "Okay, we whip around now". They actually had to filter the training data to find more moderate, calm driving scenarios to teach the model to settle down. And honestly I never would have known any different on our drive. It was very calm.
Another interesting training tidbit: we usually think about training AI by showing it "good" driving. But Rivian is also using negative reinforcement. They feed the model scenarios where human drivers messed up—specifically times when Automatic Emergency Braking (AEB) had to kick in because someone was texting or distracted. By telling the AI, "See this? Don't do this," the model actually became a safer driver.
They also taught it to handle speed bumps not by writing code that says `IF speed_bump THEN slow_down`, but by feeding it examples of humans slowing down for bumps. The system figured out the logic on its own.
The driving model isn't perfect yet, though. It struggled a bit with merging and didn't seem to fully grasp pedestrian behavior just yet.
It also seems they're still figuring out how to adequately put reigns on the AI system. They had it set up to not do right on red for this demo. Though apparently, in some other demo drives (not mine), it decided to make the turn anyway (AI has a mind of its own sometimes). I'm assuming "right on red" won't be a prohibited behavior on production versions. My guess is this was more of them just demoing the guard rails they CAN put on the AI (though they didn't ENTIRELY work), along with being cautions until they're sure the vehicle can adequately guage cross traffic.
It also seems that Rivian is getting more targeted with the data they collect. Instead of just grabbing large swaths of driving (needed early on for basic driving dynamics), they are now targeting more specific gaps in the model's behavior. The big one coming up? Parking. Currently, privacy policies stop the car from recording the first and last three minutes of a drive—which is exactly when you park. They are updating this to record those segments unless you are at home or work, so the fleet can finally learn how to navigate a parking lot.
Conclusion
If you follow the competition, this is like "FSD 12". We are basically looking at Rivian’s version of an end-to-end neural net with no hard-coded rules for learned behavior.
It’s not production-ready yet. It needs to get better at things like merges and pedestrians. But the Gen 2 vehicles have the compute power to handle it, and the foundation is solid. I can't wait to see where it goes next.
If you found this information interesting, useful, or just entertaining and you're buying a Rivian, use my referral code to get up to 500 points in the Gear Shop (and free charging!): https://rivian.com/configurations/list?reprCode=MIKE4580282
Sponsored
