Sponsored

MrMusAddict

Well-Known Member
Joined
Oct 29, 2022
Threads
36
Messages
280
Reaction score
742
Location
Oregon
Vehicles
2023 Rivian R1T
Occupation
Data Analyst
With all of the news and confusion of Gen2's refresh, there's a lot of information I thought I knew about Gen1 which I'm now realizing was incorrect (or worst case, was made incorrect retroactively).

Going down the rabbit hole, I'm learning a lot, and thought it would be helpful for me to write everything down so that the community can educate themselves and help hone in realistic expectations for autonomy. Not only for Gen1, but also Gen2.


Connecting the dots

Rivian is using Mobileye (this is something we know for certain). Mobileye is a 3rd party hardware and software autonomy provider, and we have confirmation from RJ that they intend to use Mobileye until Rivian's own hardware/software solution for autonomy is mature enough to disable/remove Mobileye.

As far as what's in the vehicles, here's what we know:

  • Gen1
  • Gen2
    • Uses Mobileye's "2Ɨ EyeQ5 High" product, also known as "Mobileye SuperVisionā„¢". This is a multi-sensor system that not only has 2 windshield-embedded cameras, but it also does have native support for 360Ā° video.
    • Rivian seems to use the samearray of other sensors, with updated resolutions on the cameras.
    • Rivian does have custom compute in Gen2: one Nvidia board with 2 processors on it. This currently is not doing anything, because it's intended to sit dormant until Rivian launches their own autonomy solution and bypass Mobileye.

So, as far as sensors go, there seems to be full parity between Gen1 and Gen2 aside from camera resolution, and the Mobileye version.

We know from other Advanced Driver Assistance Systems (ADAS) that improved camera resolution can help improve the quality of perception, but the jump from Gen1 to Gen2's resolution is not strictly necessary for perception.

This leaves us with the Mobileye hardware as being the only remaining obstacle. We now believe that Rivian has spent the last 3 years trying to connect a 360Ā° sensor suite into an incompatible perception processor. So, they upgraded Gen2 to Mobileye's new processor so that they can start providing autonomy until they launch their own solution. We have evidence to support this;


Inferences

On the Mobileye product page for SuperVision, they specifically call out "Full surround high-definition computer vision perception". What this tells me is that Mobileye very intentionally locks down their features based on their own hardware specifications. No high-def, no SuperVision compatibility. The camera resolution is an obstacle because Mobileye is an obstacle.

So although Gen1 cameras are sufficient for a custom autonomy solution, Gen1 has no custom compute. Instead, they relied on Mobileye and expected to link Rivian's sensors. Unfortunately, they are locked into the safety/compatibility standards set by Mobileye, and Mobileye has deemed Gen1 cameras as insufficient for SuperVision compatibility.

Gen2 has custom compute hardware, but this is to future-proof the vehicles for Rivian's own Mobileye-alternative if and when they can finally get it up and running themselves.

For now, all Rivians use Mobileye under the hood, just dressed up with Rivian's aesthetics;
  • Gen1 is limited by Mobileye to Forward Facing Perception and will never get Rivian's custom perception.
  • Gen2 is greenlit by Mobileye for 360Ā° Perception, and may eventually get Rivian's custom perception.

What can we do about it?

I guess that's up to you guys. I'd like to see some change, but I'm just here to educate the community on the past communications and old-vs-new hardware, as well as Rivian's inferred bind with Mobileye's restrictions.

Wassym has said that they are not looking into hardware retrofits at this time. If my understanding of the obstacles are correct, there are 2 retrofits are necessary for the initial RAP+ launch, and one additional retrofit for Rivian's eventual perception compute replacement.

  • Necessary for RAP+ (mandated by Mobileye)
    • Upgraded Cameras
    • Replacement Mobileye SOC
  • Additionally necessary for Rivian's perception compute
    • Nvidia board

I can understand Wassym's pushback on offering these retrofits; at this point we'd effectively be asking to replace the car's nervous system. So if that's what the community wants to push for, just keep that in mind (and the surcharge for the upgrade would likely be astronomical as a result).


Looking Forward/Summary

The community has been led to believe that Rivian has been developing and implementing their own autonomy solution this whole time. In reality, everything we've seen has been thanks to Mobileye. We have not yet had a taste of Rivian's own autonomy capabilities.

For Gen1 Vehicles, it is realistic to expect no new autonomous features being added to Driver+. Wassym did say Rivian's philosophy is to support new features on Gen1 if the hardware allows it. It's looking like it never will.

For Gen2 Vehicles, it is realistic to only expect new features that align with the product page for Mobileye SuperVision. Although Gen2 is equipped with additional compute power intended for Rivian's custom compute, we have not yet had any proof that Rivian can develop a competitive autonomy solution on their own. Rivian will be launching Mobileye's advertised features throughout 2024, meaning you will likely not see Rivian's solution until well into 2025 or later, if they're even capable of launching it.
Sponsored

 
Last edited:

DuoRivian

Well-Known Member
Joined
Sep 3, 2023
Threads
2
Messages
735
Reaction score
881
Location
California
Vehicles
Rivian R1T and an R1S
Occupation
IT
Clubs
 
Thanks for this comprehensive overview. My question is - should Rivian be spending money on developing their own autonomy when they can (and do) use a third party. They are never going to be class leading - too many big players in this area and the money saved could be used for service centers, RANs, etc.
 
OP
OP
MrMusAddict

MrMusAddict

Well-Known Member
Joined
Oct 29, 2022
Threads
36
Messages
280
Reaction score
742
Location
Oregon
Vehicles
2023 Rivian R1T
Occupation
Data Analyst
Thanks for this comprehensive overview. My question is - should Rivian be spending money on developing their own autonomy when they can (and do) use a third party. They are never going to be class leading - too many big players in this area and the money saved could be used for service centers, RANs, etc.
If I were a short-term investor, I would say no. If I were a long-term investor, I would say probably.

They've proven capable as a vertical hardware integrator, but I'm feeling pretty duped on the software side. Granted, all of the software for the UX is Rivian's, but the perception on Autonomy has been Mobileye this whole time. So, now I'm realizing that Rivian is essentially 3 years behind where I thought they were.

They could have their own tricks up their sleeves, like relying on fleet data (despite no Rivian compute on the road) to train an internal model. So perhaps not as far behind as I'm thinking.
 

Mellowyellow

Well-Known Member
Joined
Jan 27, 2024
Threads
6
Messages
407
Reaction score
599
Location
Westchester NY
Vehicles
R1S
Clubs
 
Isnā€™t the software google android customized by rivian?

Also great summary!
 
Last edited:
OP
OP
MrMusAddict

MrMusAddict

Well-Known Member
Joined
Oct 29, 2022
Threads
36
Messages
280
Reaction score
742
Location
Oregon
Vehicles
2023 Rivian R1T
Occupation
Data Analyst
Isnā€™t the software google android customized by rivian?
The operating system is Rivian's own making, yes. My post is purely talking about autonomous driving compute though. So far, Mobileye's been the one actually processing all the perception and driving the vehicle during Driver+. What I'm inferring is that the only involvement Rivian has with Driver+ at this point is the UI, and integrating/validating the sensors into the Mobileye processor.
 

Sponsored

Dark-Fx

Well-Known Member
First Name
Brian
Joined
Jul 15, 2020
Threads
126
Messages
11,590
Reaction score
22,961
Location
Michigan
Vehicles
R1T, R1S, Livewire One, Fisker Ocean, Sierra EV
Occupation
Engineering
Clubs
 
Response to speculation with more speculation while acting authoritative. Bleh.
 

bdwalters

Well-Known Member
First Name
Brian
Joined
Feb 12, 2024
Threads
3
Messages
148
Reaction score
243
Location
Austin, TX
Vehicles
R1S, Model 3, Model Y
Occupation
Hardware Engineer
I just donā€™t think Rivian has the R&D budget to do autonomy. It costs a fortune. Even in the Gen 2, I wonder if all of that expensive hardware is just a waste of money that will never be used. They should consider licensing Teslaā€™s system.
 

Zoidz

Well-Known Member
First Name
Gil
Joined
Feb 28, 2021
Threads
160
Messages
4,268
Reaction score
9,409
Location
PA
Vehicles
23 R1S Adv, Avalanche, BMWs-X3,330cic,K1200RS bike
Occupation
Engineer
Interesting information. OP uses phrases such as ā€œwe believeā€, ā€œinferredā€, ā€œseems to beā€. Take the information for what it is worth, buit it is still one personā€™s interpretation and speculation at the end of the day.

It seems some people want to torch, sue, or burn Rivian at the stake based on the belief that Rivian mislead us intentionally. Iā€™ll offer another possibility - that the OEM told Rivian it was possible, and as the integration project progressed, it became clear that there were obstacles that could not be overcome with the Gen 1 hardware.

Iā€™m not trying to defend Rivian, just offering this as a possible scenario. I experienced this personally a number of years ago. Allen Bradley stated that a paticular processor and ethernet interface was capable of a certain performance level. We used that configuration for a critical project, only to find out after 4 months of development and performance stress testing that neither the processor nor the Ethernet interface was capable.
 
OP
OP
MrMusAddict

MrMusAddict

Well-Known Member
Joined
Oct 29, 2022
Threads
36
Messages
280
Reaction score
742
Location
Oregon
Vehicles
2023 Rivian R1T
Occupation
Data Analyst
Interesting information. OP uses phrases such as ā€œwe believeā€, ā€œinferredā€, ā€œseems to beā€. Take the information for what it is worth, buit it is still one personā€™s interpretation and speculation at the end of the day.

It seems some people want to torch, sue, or burn Rivian at the stake based on the belief that Rivian mislead us intentionally. Iā€™ll offer another possibility - that the OEM told Rivian it was possible, and as the integration project progressed, it became clear that there were obstacles that could not be overcome with the Gen 1 hardware.

Iā€™m not trying to defend Rivian, just offering this as a possible scenario. I experienced this personally a number of years ago. Allen Bradley stated that a paticular processor and ethernet interface was capable of a certain performance level. We used that configuration for a critical project, only to find out after 4 months of development and performance stress testing that neither the processor nor the Ethernet interface was capable.
To your point of Mobileye possibly over promising their hardware's functionality, that actually aligns with the messaging of my post. Ultimately, I'm not saying Rivian acted with malice, or even incompetence. Rather, Mobileye is the obstacle, whether for to mandates, or their own possible incompetence as Rivian's partner.

I'll say it's definitely more than one person's interpretation. This post is a consolidation of thoughts and observations from other people who've dabbled down this same rabbit hole. Sure I individually made this post, but there's a lot of corroboration from other people's understandings.
 

BKR1S

Member
Joined
Apr 17, 2022
Threads
0
Messages
11
Reaction score
6
Location
Brooklyn, NY
Vehicles
Tesla Model Y
With all of the news and confusion of Gen2's refresh, there's a lot of information I thought I knew about Gen1 which I'm now realizing was incorrect (or worst case, was made incorrect retroactively).

Going down the rabbit hole, I'm learning a lot, and thought it would be helpful for me to write everything down so that the community can educate themselves and help hone in realistic expectations for autonomy. Not only for Gen1, but also Gen2.


Connecting the dots

Rivian is using Mobileye (this is something we know for certain). Mobileye is a 3rd party hardware and software autonomy provider, and we have confirmation from RJ that they intend to use Mobileye until Rivian's own hardware/software solution for autonomy is mature enough to disable/remove Mobileye.

As far as what's in the vehicles, here's what we know:

  • Gen1
  • Gen2
    • Uses Mobileye's "2Ɨ EyeQ5 High" product, also known as "Mobileye SuperVisionā„¢". This is a multi-sensor system that not only has 2 windshield-embedded cameras, but it also does have native support for 360Ā° video.
    • Rivian seems to use the samearray of other sensors, with updated resolutions on the cameras.
    • Rivian does have custom compute in Gen2: one Nvidia board with 2 processors on it. This currently is not doing anything, because it's intended to sit dormant until Rivian launches their own autonomy solution and bypass Mobileye.

So, as far as sensors go, there seems to be full parity between Gen1 and Gen2 aside from camera resolution, and the Mobileye version.

We know from other Advanced Driver Assistance Systems (ADAS) that improved camera resolution can help improve the quality of perception, but the jump from Gen1 to Gen2's resolution is not strictly necessary for perception.

This leaves us with the Mobileye hardware as being the only remaining obstacle. We now believe that Rivian has spent the last 3 years trying to connect a 360Ā° sensor suite into an incompatible perception processor. So, they upgraded Gen2 to Mobileye's new processor so that they can start providing autonomy until they launch their own solution. We have evidence to support this;


Inferences

On the Mobileye product page for SuperVision, they specifically call out "Full surround high-definition computer vision perception". What this tells me is that Mobileye very intentionally locks down their features based on their own hardware specifications. No high-def, no SuperVision compatibility. The camera resolution is an obstacle because Mobileye is an obstacle.

So although Gen1 cameras are sufficient for a custom autonomy solution, Gen1 has no custom compute. Instead, they relied on Mobileye and expected to link Rivian's sensors. Unfortunately, they are locked into the safety/compatibility standards set by Mobileye, and Mobileye has deemed Gen1 cameras as insufficient for SuperVision compatibility.

Gen2 has custom compute hardware, but this is to future-proof the vehicles for Rivian's own Mobileye-alternative if and when they can finally get it up and running themselves.

For now, all Rivians use Mobileye under the hood, just dressed up with Rivian's aesthetics;
  • Gen1 is limited by Mobileye to Forward Facing Perception and will never get Rivian's custom perception.
  • Gen2 is greenlit by Mobileye for 360Ā° Perception, and may eventually get Rivian's custom perception.

What can we do about it?

I guess that's up to you guys. I'd like to see some change, but I'm just here to educate the community on the past communications and old-vs-new hardware, as well as Rivian's inferred bind with Mobileye's restrictions.

Wassym has said that they are not looking into hardware retrofits at this time. If my understanding of the obstacles are correct, there are 2 retrofits are necessary for the initial RAP+ launch, and one additional retrofit for Rivian's eventual perception compute replacement.

  • Necessary for RAP+ (mandated by Mobileye)
    • Upgraded Cameras
    • Replacement Mobileye SOC
  • Additionally necessary for Rivian's perception compute
    • Nvidia board

I can understand Wassym's pushback on offering these retrofits; at this point we'd effectively be asking to replace the car's nervous system. So if that's what the community wants to push for, just keep that in mind (and the surcharge for the upgrade would likely be astronomical as a result).


Looking Forward/Summary

The community has been led to believe that Rivian has been developing and implementing their own autonomy solution this whole time. In reality, everything we've seen has been thanks to Mobileye. We have not yet had a taste of Rivian's own autonomy capabilities.

For Gen1 Vehicles, it is realistic to expect no new autonomous features being added to Driver+. Wassym did say Rivian's philosophy is to support new features on Gen1 if the hardware allows it. It's looking like it never will.

For Gen2 Vehicles, it is realistic to only expect new features that align with the product page for Mobileye SuperVision. Although Gen2 is equipped with additional compute power intended for Rivian's custom compute, we have not yet had any proof that Rivian can develop a competitive autonomy solution on their own. Rivian will be launching Mobileye's advertised features throughout 2024, meaning you will likely not see Rivian's solution until well into 2025 or later, if they're even capable of launching it.
Thanks, this is very thoughtful. Iā€™ve always wondered why the rendering could never show vehicles around the car (other than in front) despite having sensors for it. And would explain why they canā€™t get the auto lane change to work.
 

Sponsored

CharonPDX

Well-Known Member
First Name
Charon
Joined
Jul 12, 2021
Threads
24
Messages
1,627
Reaction score
2,561
Location
Cascadia
Vehicles
R1T LE, Mach-E, Arcimoto FUV, Repl: F-250/Model S
Occupation
InfoSec Geek
Clubs
 
Isnā€™t the software google android customized by rivian?

Also great summary!
Please note that the infotainment system is the Google-base OS. That is almost certainly *NOT* what is running the actual battery/motor control system, nor the driver-assistance systems.

Those tend to be *COMPLETELY* separate compute systems inside a vehicle, with the non-infotainment having *MUCH* simpler ā€œoperating systemsā€œ that are also far more robust.

Iā€™m sure plenty of long-time owners have had the infotainment system spontaneously reset itself on them. If it were the same ā€œOSā€ that was running the vehicle, that would mean the vehicle would have just shut down on the road. That didnā€™t happen.

The safety requirements for actual ā€œvehicle controlā€ are MUCH higher than for ā€œwhat shows on the display.ā€

Also look up ā€œreal time OSā€ (both the specific OS named ā€œReal Time OSā€ and the general concept that the specific-named OS is an implementation of.) Vehicle control systems *MUST* be fully real-time. Nothing can ever ā€œstallā€ or ā€œhangā€ the OS. *EVER*. That isnā€™t necessary with the displays/UI. If the speedometer takes an extra 50 milliseconds to change form 45 to 46 MPH, thatā€™s no big deal. If the touchscreen takes an extra 100 milliseconds to respond to the touch for heated seat control, thatā€™s not a big deal. If the map takes an extra 500 milliseconds to switch from dark mode to light mode, thatā€™s no big deal.

If the throttle input system takes an extra 10 milliseconds to respond to someone mashing it to the floor to avoid an accident being rear-ended by a semi - that *IS* a big deal.

Googleā€˜s Android Automotive based offerings arenā€™t up to that task. Even Google only says itā€™s for ā€œIn Vehicle Infotainmentā€ systems, not actually running the car.
 

IGR

Well-Known Member
Joined
Jul 7, 2023
Threads
7
Messages
273
Reaction score
384
Location
SoCal
Vehicles
Rivian R1S
Occupation
I make things
Clubs
 
They could have their own tricks up their sleeves, like relying on fleet data (despite no Rivian compute on the road) to train an internal model. So perhaps not as far behind as I'm thinking.
I'm sure the gigabytes of data Rivian vehicles upload each night is the "fleet data" you are referring to, I hope they put it to good use at least.

Based on this information I would assume Gen2 Rivains should be able to set the Driver+ / cruise control higher than 85 mph? I can imagine those are Mobileye hardware limitation as well?
 

GHuff

Well-Known Member
First Name
John
Joined
Feb 25, 2021
Threads
4
Messages
191
Reaction score
280
Location
Tyler
Vehicles
Tesla
As a R1 owner, I'm very disappointed. Rivian has definitely led us to believe that what we have today is not the end product, yet it sounds like it is.

I'm fine and completely understand not having autonomous driving features. But not having normal adaptive cruise control that isn't tied to specific roads is very disappointing.

Also, not having 360 awareness while driving is frustrating. The driver dashboard is basically worthless (because it will forever only render objects in front) and it is such a main focal point of the vehicle. I had hopes that this would improve over time. These renderings are basically the entire purpose of the driver dash--yet it effectively useless? That's a tough pill to swallow. I think it's safe to assume that Rivian had a grander vision for this dashboard, the renderings, and its use. Only rendering objects in front of the vehicle is a bit underwhelming for a computer on wheels.

EDIT: apparently normal adaptive cruise control does exist. That's news to me.
 
Last edited:

Zoidz

Well-Known Member
First Name
Gil
Joined
Feb 28, 2021
Threads
160
Messages
4,268
Reaction score
9,409
Location
PA
Vehicles
23 R1S Adv, Avalanche, BMWs-X3,330cic,K1200RS bike
Occupation
Engineer
Please note that the infotainment system is the Google-base OS. That is almost certainly *NOT* what is running the actual battery/motor control system, nor the driver-assistance systems.

Those tend to be *COMPLETELY* separate compute systems inside a vehicle, with the non-infotainment having *MUCH* simpler ā€œoperating systemsā€œ that are also far more robust.

Iā€™m sure plenty of long-time owners have had the infotainment system spontaneously reset itself on them. If it were the same ā€œOSā€ that was running the vehicle, that would mean the vehicle would have just shut down on the road. That didnā€™t happen.

The safety requirements for actual ā€œvehicle controlā€ are MUCH higher than for ā€œwhat shows on the display.ā€

Also look up ā€œreal time OSā€ (both the specific OS named ā€œReal Time OSā€ and the general concept that the specific-named OS is an implementation of.) Vehicle control systems *MUST* be fully real-time. Nothing can ever ā€œstallā€ or ā€œhangā€ the OS. *EVER*. That isnā€™t necessary with the displays/UI. If the speedometer takes an extra 50 milliseconds to change form 45 to 46 MPH, thatā€™s no big deal. If the touchscreen takes an extra 100 milliseconds to respond to the touch for heated seat control, thatā€™s not a big deal. If the map takes an extra 500 milliseconds to switch from dark mode to light mode, thatā€™s no big deal.

If the throttle input system takes an extra 10 milliseconds to respond to someone mashing it to the floor to avoid an accident being rear-ended by a semi - that *IS* a big deal.

Googleā€˜s Android Automotive based offerings arenā€™t up to that task. Even Google only says itā€™s for ā€œIn Vehicle Infotainmentā€ systems, not actually running the car.
Rivian RTOS is Blackberry QNX.
 

Tim-in-CA

Well-Known Member
Joined
Sep 30, 2021
Threads
42
Messages
1,609
Reaction score
3,061
Location
So Cal
Vehicles
šŸŒ²šŸŒ²R1S, Lucid Air, T-Bird
I'm fine and completely understand not having autonomous driving features. But not having normal adaptive cruise control that isn't tied to specific roads is very disappointing.
Adaptive cruise control works on all roads. Adaptive cruise is when the vehicle slows to match the speed of the vehicle ahead if set speed is higher than the vehicle in front. What only works on mapped roads (pretty much highways/interstates/freeways) is adaptive cruise with auto steer where the vehicle will center itself and follow lanes. A single downward "click" of the drive stalk activates adaptive cruse, a double downward click activates Driver+ ... I definitely agree that we G1 users are stuck with pretty basic features and were fed that more was to come. Oh well.
Sponsored

 
 





Top