mpshizzle
Well-Known Member
- Thread starter
- #1
This stuff is a lot easier to show in a video, and it shows much more detail throughout the process. If you want to check that out, I uploaded under Thunder Volt on Youtube:
Otherwise, if you'd rather read about it, you can stick with the written version
A couple of weeks ago I posted about Rivian moving off Mobileye and onto their own autonomy stack. There was a lot of interest in that post, so I figured I would continue to share my testing.
Since then, I wanted to answer the golden question for autonomy: cameras only, or cameras + radar? We know Rivian uses both, but how much and what for? So, I went out and ran a series of experiments to figure out exactly how Rivian’s driver assistance system is leveraging the vehicle’s full sensor suite. The plan is to cover cameras, and then radars to see what changes.
But First - Where are the Cameras and Radars?
• Windshield cameras: Two Rivian units (wide and narrow) + the now-deprecated Mobileye camera.
• Bumper cameras: Front and rear bumper mounted cams.
• Mirror cameras: Each mirror has three – forward-facing, rear-facing, and downward-facing.
• Radar: Five total – one imaging radar dead-center in the front fascia (used heavily for ACC), plus four corner radars (front + rear). The imaging radar can tell distance and exact position in X/Y.
Baseline Run
I first drove around with everything uncovered to get a baseline. In a busy Walmart parking lot (great torture test for the visualizations), the driver display showed vehicles, pedestrians, and angles of vehicles really well.
What stood out: in stop-and-go situations, the system often showed multiple cars queued ahead — something you usually only see on radar-equipped vehicles. This hints that radar is indeed actively feeding the visuals, not just cameras.
Camera Block Test
Next, I taped over all cameras except the old Mobileye unit (just in case Rivian left it as a fallback).
• Result: The display was chaotic. Objects flickered, misidentified vehicle types everywhere, vehicles shaking around onscreen. The Mobileye unit clearly isn’t in play anymore — not even for visualization. Though I did end up covering it just to be sure, but behavior didn't change at all.
• Warnings: The system complained about blocked windshield cameras. In some of my earlier testing I also got warnings for blocked mirror cameras (this is new with this software version). But the mirror camera warning seems to be inconsistent. With those warnings active, Highway Assist and ACC were disabled.
• Weird observation: Even with every camera taped, the system hallucinated lane lines on a road that had none — and they were eerily accurate, following the curvature of the road. Either Rivian is fusing in map data or the radars are estimating road geometry.
When I uncovered just the Rivian windshield cameras, all the warnings disappeared and visuals stabilized (at least for vehicles within the windshield camera's view). Highway Assist worked again, proving that the Rivian autonomy stack now leans very heavily on its own front cameras for steering. Interestingly though, it exhibited a bit of erratic behavior with one (but not all) of the lane changes. It also lost track of the lanes completely in a spot that is very clearly marked. My takeaway: For driving, the radars do the heavy lifting for awareness of surroundings. But that doesn't mean the surround cameras go unused. Though they play a lesser role, they clearly do contribute. The windshield cam on the other hand, absolutely is vital.
Radar Block Test
Next, I covered all the radars using rags with aluminum foil wrapped inside. (I had lots of people staring in a very bad way haha)
• At first, no warnings appeared and ACC still engaged. Eventually, though, the system threw a “radar blocked” warning.
• Despite that, visuals remained excellent. The driver display still identified vehicles correctly and with stability, relying primarily on the cameras.
So, the system is clearly camera-primary for visualization, with radar providing redundancy and confidence.
Takeaways
What do you all think?
Otherwise, if you'd rather read about it, you can stick with the written version
A couple of weeks ago I posted about Rivian moving off Mobileye and onto their own autonomy stack. There was a lot of interest in that post, so I figured I would continue to share my testing.
Since then, I wanted to answer the golden question for autonomy: cameras only, or cameras + radar? We know Rivian uses both, but how much and what for? So, I went out and ran a series of experiments to figure out exactly how Rivian’s driver assistance system is leveraging the vehicle’s full sensor suite. The plan is to cover cameras, and then radars to see what changes.
But First - Where are the Cameras and Radars?
• Windshield cameras: Two Rivian units (wide and narrow) + the now-deprecated Mobileye camera.
• Bumper cameras: Front and rear bumper mounted cams.
• Mirror cameras: Each mirror has three – forward-facing, rear-facing, and downward-facing.
• Radar: Five total – one imaging radar dead-center in the front fascia (used heavily for ACC), plus four corner radars (front + rear). The imaging radar can tell distance and exact position in X/Y.
Baseline Run
I first drove around with everything uncovered to get a baseline. In a busy Walmart parking lot (great torture test for the visualizations), the driver display showed vehicles, pedestrians, and angles of vehicles really well.
What stood out: in stop-and-go situations, the system often showed multiple cars queued ahead — something you usually only see on radar-equipped vehicles. This hints that radar is indeed actively feeding the visuals, not just cameras.
Camera Block Test
Next, I taped over all cameras except the old Mobileye unit (just in case Rivian left it as a fallback).
• Result: The display was chaotic. Objects flickered, misidentified vehicle types everywhere, vehicles shaking around onscreen. The Mobileye unit clearly isn’t in play anymore — not even for visualization. Though I did end up covering it just to be sure, but behavior didn't change at all.
• Warnings: The system complained about blocked windshield cameras. In some of my earlier testing I also got warnings for blocked mirror cameras (this is new with this software version). But the mirror camera warning seems to be inconsistent. With those warnings active, Highway Assist and ACC were disabled.
• Weird observation: Even with every camera taped, the system hallucinated lane lines on a road that had none — and they were eerily accurate, following the curvature of the road. Either Rivian is fusing in map data or the radars are estimating road geometry.
When I uncovered just the Rivian windshield cameras, all the warnings disappeared and visuals stabilized (at least for vehicles within the windshield camera's view). Highway Assist worked again, proving that the Rivian autonomy stack now leans very heavily on its own front cameras for steering. Interestingly though, it exhibited a bit of erratic behavior with one (but not all) of the lane changes. It also lost track of the lanes completely in a spot that is very clearly marked. My takeaway: For driving, the radars do the heavy lifting for awareness of surroundings. But that doesn't mean the surround cameras go unused. Though they play a lesser role, they clearly do contribute. The windshield cam on the other hand, absolutely is vital.
Radar Block Test
Next, I covered all the radars using rags with aluminum foil wrapped inside. (I had lots of people staring in a very bad way haha)
• At first, no warnings appeared and ACC still engaged. Eventually, though, the system threw a “radar blocked” warning.
• Despite that, visuals remained excellent. The driver display still identified vehicles correctly and with stability, relying primarily on the cameras.
So, the system is clearly camera-primary for visualization, with radar providing redundancy and confidence.
Takeaways
- As it stands Rivian is definitely utilizing both cameras and radars heavily. The driver screen visualization seems to rely more on cameras, with radars filling in while the actual driving seems to rely more on radar with cameras filling in. Except for the windshield cameras. That's vital for both.
- Having multiple kinds of sensors allows for a bit of redundancy. When mud or snow gets caked on one or two of the cameras and/or radars, it doesn't completely cripple the system, since it has other sensors to rely on.
- Unlike Tesla (which relies on massive fleet data to make vision-only work), Rivian has a much smaller fleet and is compensating with high-quality multimodal sensors.
- Future potential: Based on the hallucinated lane lines and perception-first behavior, Rivian is clearly moving toward a system that will eventually go beyond highway geo-locks and adapt better to real-world conditions. Once Rivian's software has matured enough and they have collected enough high quality training data, I could see a future where they go camera only. If they did so, they'd loose the redundancy, but it would be slightly cost adventageous. Only time will tell if they decide to go this route.
What do you all think?
Sponsored
