The biggest question here is whether Tesla’s operating model for autonomous driving requires other data than cameras? Can excessive so-called sensor fusion data impair a computer’s decision-making in fast situations? I don’t know the right answer to these questions myself, but I trust Tesla’s expertise in their solution.
2 Likes
In the real world, there can indeed be situations where removing a single sensor from a system can improve the outcome in the short term. Increasing the number of new sensors also doesn’t always bring enough benefits compared to the costs, and increasing the complexity of the system naturally has a price, and sometimes it creates new problems that simpler systems don’t have. As a general rule, however, in AI work, more data is better than less data, especially if these data sources are fundamentally different from each other. The opposite argument is mainly encountered only among Tesla investors.
Imagine you are a Master of Science in Engineering (diplomi-insinööri) and Musk has hired you to build a 100% camera-based system for Tesla that informs the driver about driving conditions and provides recommendations based on them. You would then have a professional obligation to tell Musk that his desired system can indeed be built and would certainly be quite functional, but the AI could be improved by adding other sensors to the system in addition to the camera.
For example, a temperature sensor and a humidity sensor could support the observations made by the camera. Or even simply fetching local weather forecasts from a weather forecasting company’s website and taking this information into account when the model interprets the image produced by the camera about the surrounding weather conditions. So, while you could in principle build a 100% camera-based system for this purpose, very simple additions can have a big impact on the end result.
A fully camera-based system is definitely a tremendously good solution for many basic driving situations, but when driving, situations arise where a camera is weaker than another sensor and, at worst, completely useless, for example, in dense fog. Because the impact of errors on personal safety when driving is enormous, these cannot be dismissed with a shrug, stating that the car system will stay on the road most of the time. Unless the driving environment is restricted, for example, to a fenced airport area, or autonomous driving is stopped in conditions unfavorable to the camera.
Even if the camera remains the primary way to manage an autonomously driving car, adding a new sensor is a logical solution for situations where the data produced by the camera sensor is of poor quality and unsuitable. Of course, if ideological reasons prevent this, then blood, sweat, and tears are ahead when trying to force a non-functional tool to solve a problem for which it was not intended.
30 Likes
Who now has more (good) data and who has less? If Tesla has pure camera data from its entire FSD-fleet, including a couple of million cars, and Waymo has sensor noise data from 1500 cars (priced at 150k), which one is in a better position to start training a machine? Tesla uses LIDAR to verify robotaxi training, so camera data decisions can be challenged and retrained.
If the AI’s training base consists of all raw sensor data from different types of sensors in a commensurable way, nothing good will come of it. As I understand it, Waymo uses traditional algorithms (Kalman Filters) in lower-level sensor fusion, and then once the “situational truth” is formed, the AI is trained only at this higher level, and thus traffic objects can be classified.
I personally consider it possible that Tesla will encounter a dangerous edge-case in Vision, where in robotaxi validation, LIDAR (or radars) show a situation that cameras cannot possibly show or be trained for. Will the consequence of such a situation then be a phantom braking, a new HW version, or the collapse of the entire business case? That remains to be seen.
1 Like
Why should one even be able to drive in dense fog (or other conditions that are difficult/dangerous even for humans)? Or conversely: why couldn’t a camera-based system drive at least in the same conditions as a human?
Cameras and AI seem to be Tesla’s chosen path even for Optimus, come what may: Inside the Strategy Shift at Optimus, Tesla's Humanoid Robot Program - Business Insider
2 Likes
The question, however, is what the cameras "see" and how the software is able to make correct decisions from that. As for the amount of different information and its potential drawback, there are also more than one camera in the car.
Well, sometimes you just have to drive in any weather. This was probably the dumbest argument read so far.
The fact is that a camera alone is not enough. I spend enough time in traffic that there are situations where cameras just can’t see. But a human can see. It’s enough to take a current Tesla and drive off during the morning dew. The car is blind when the cameras are fogged up.
24 Likes
If this happens, it’s actually a self-inflicted problem. Tesla would have the opportunity to collect data from other sensors for storage and training, if those sensors had been installed in the cars in the first place. Musk has many times gone all-in on a certain technology and won big that way, but since Tesla has long been a market leader in autonomous driving, I don’t really understand why such significant technological risks should be taken from a clear advantageous position. It’s usually worth playing ‘rich or poor’ if you don’t have much to lose. Of course, if this succeeds, the reward is also incredibly high, so it truly remains to be seen 
Of course, this can be done by stating that Tesla is autonomously undrivable in certain conditions. If this is due to saving on some sensor, it doesn’t seem like a smart long-term strategy. Fog is a good example of an unequivocally impossible problem for cameras to solve. There are, of course, other examples between fog and perfect driving conditions, but that should suffice to demonstrate the problems that full-camera systems encounter.
In my opinion, directly replicating human driving ability one-to-one is not a meaningful approach to autonomous driving. For an autonomous car, it is possible to design a car with superior capabilities compared to human senses. An example would be radar waves, which the human body cannot perceive. With these superhuman capabilities, we can then compensate for the fact that we cannot install a self-learning supercomputer capable of modeling the environment like a brain in every car. For now, we have to make do with a significantly less powerful AI processor, which should nevertheless be comparable to human driving ability or, preferably, better in selected situations.
15 Likes
The claim is that the performance is significantly better than a human’s because cameras see in every direction 24/7 better than a human, and AI doesn’t get tired, nervous, or intoxicated.
This is what all Tesla discussions nowadays boil down to. A “yes-no” argument until it somehow turns out that the solution simply doesn’t work, or it does work and money comes in truckloads.
Speaking of trucks, it’ll be interesting to see what sensors the Semi includes. Truck braking distances, especially on icy roads, are so long that I wonder if vision-only will be enough.
FSD has driven billions and billions of kilometers. If morning dew were a problem, one would imagine YouTube would be full of “My Tesla can’t handle morning dew” videos.
3 Likes
Indeed, this has been done, and previously, fusion and Kalman algorithms have also been used. New Model S/X include radar, and FSD usage could be compared to radar data. If an intervention occurred, one can compare from the radar data whether the system would have managed without intervention using radar. Musk has said that radar has not been beneficial, and therefore they have turned it off. However, if someone demonstrates otherwise, for example, in certain weather conditions, it’s easy for Tesla to turn on the radars in new S/X FSD cars and verify the situation in practice.
Tesla’s way of developing technology is to have teams compete with different approaches. Therefore, it is likely that LIDAR and radar teams have been involved in the competition over the years to achieve better results than the camera team has.
Waymo has already responded to the earlier claim that AI would perform better with a larger number of sensors and that only Tesla investors would dispute it. Waymo has been continuously reducing the number of sensors. Waymo also does not use AI in low-level sensor fusion.
2 Likes
I’ve been thinking about this too. We use AI a lot at work; its goal is to “find a needle in a haystack” as accurately as possible. I’ve noticed that for most people, the starting point would be to add more and more data to look for that needle.
In reality, sometimes we have achieved much better results with two models, one of which tries to identify “is it the needle” and the other tries to prove that it is not.
1 Like
It’s not impossible because Tesla can’t do it
Part of the problem here appears to be that Musk thinks something doesn’t work because Tesla can’t make it work, and he doesn’t want to admit that others are solving the sensor fusion problem.
Tesla simply couldn’t solve sensor fusion, so it focused on achieving autonomy solely through camera vision. However, those who continued to work on the issue have made significant progress and are now reaping the rewards.
Waymo and Baidu, both of which have level 4 autonomous driving systems currently commercially operating without supervision, unlike Tesla, have heavily invested in sensor fusion.
Amir Husain, an AI entrepreneur who sits on the Boards of Advisors for IBM Watson and the Department of Computer Science at UT Austin, points to advancements in the use of Kalman filters and Bayesian techniques to solve sensor noise covariance.
He commented on Musk’s statement regarding the use of radar and lidar sensors:
The issue isn’t a binary disagreement between two sensors. It generates a better estimate than any individual sensor can produce on its own. They all have a margin of error. Fusion helps reduce this.
If Musk’s argument held, why would the human brain use eyes, ears, and touch to estimate object location? Why would aircraft combine radar, IRST, and other passive sensors to estimate object location? This is a fundamental misunderstanding of information theory. Every channel has noise. But redundancy reduces uncertainty.
Musk’s main argument to focus on cameras and neural nets has been that the roads are designed for humans to drive and humans drive using their eyes and brain, which are the hardware and software equivalent of cameras (eyes) and neural nets (brain).
Now, most other companies developing autonomous driving technologies are also focusing on this, but to surpass humans and achieve greater levels of safety through precision and redundancy, they are also adding radar and lidar sensors to their systems.
30 Likes
Fred Lambert typically presents things in a Tesla-hostile manner, so half of the story is missing. The article contained Lambert’s DM exchange with Musk, in which Musk stated that perhaps a High-Res radar with vision could be a better option. Now, that High-Res radar has since been tested in Model S/X cars and it wasn’t found to perform better than cameras, even though the latest Kalman development steps have been published in scientific articles.
Musk’s comments are his marketing for robotaxi safety, and Tesla uses LIDAR and radars for research purposes, even though Lambert believes Musk has “forbidden everything but cameras” at Tesla. As the expansion of robotaxis is a kind of race between Waymo and Tesla, we will now see if a solution based on sensor fusion, expensive sensor cars, and HD mapping can expand faster than Tesla’s robotaxi.
“Reaping the rewards” is perhaps a strong statement if billions of dollars are being spent on Waymo and LIDAR/radar/cameras with new Kalman filters don’t see a bike lane as illegal parking or warn the customer about approaching cyclists - or keep doors locked until it’s safe to exit.
5 Likes
I’m also annoyed by Lambert’s very selective way of reporting things against Tesla, but “reaping the rewards” was quite literally well justified in the following paragraph:
“Waymo and Baidu, both of which have level 4 autonomous driving systems currently commercially operating without supervision, unlike Tesla, have heavily invested in sensor fusion.”
1 Like
And further to that cyclist’s case: the cyclist was injured and wants money from Waymo/Alphabet. The article is based purely on the cyclist’s account. Apparently, in that block, immediately after a short bike lane, there’s a “yellow zone” where taxis are allowed to stop briefly. Furthermore, Waymo’s safety system does not lock the doors but warns the passenger with an audible signal about an approaching cyclist. Nowhere has it even been claimed that the system did not function correctly.
1 Like
Fred Lambert’s views on the matter were not the point of the writing, but rather the bolded part.
So, Tesla didn’t have the expertise to make HW4 radars work better.
Do you have any other information about this case than the cyclist’s perspective? And do Lyfts, Ubers, and Robotaxis really keep their doors locked so that the car/driver decides when the customer can exit?
2 Likes
Mark Spiegel’s Stanphyl Capital investor letter. Scam markets and failed Tesla shorts:
2 Likes
The man is a professional at destroying investors’ money. With a track record like that (0.9% annual return 2011-2025), one can only wonder who invests money in such a fund. Presumably, there have been a few strong moments in history when they were short at a big dip, but in the long run…
Bears… can be right for years and still lose money hand over fist. I don’t fully understand Tesla’s performance either, but at least I understand that it cannot be shorted in any way, day trading excluded.
7 Likes
Washington postissa oli mielenkiintoinen Tesla onnettomuus ja oikeus juttu.
Luettavissa arkistosta:
https://archive.ph/s1psp
Tesla said it didn’t have key data in a fatal crash. Then a hacker found it.
Erittäin huonoa mainosta brändille tällaiset jutut ja keissit joka tapauksessa.
Already, the impact of Tesla’s loss is reverberating beyond the Miami courtroom.
A shareholder lawsuit in Texas filed this month alleges Tesla defrauded investors with its promotion of autonomous driving technology. The shareholder complaint specifically cites the outcome of the Florida case before accusing Tesla of “wrongful acts and omissions.” Experts said the verdict has also given fresh momentum to pending cases across the country, including one expected to go to trial in Northern California in the fall over the death of a 15-year-old.
7 Likes
Tesla’s new master plan v4 was unveiled a couple of days ago: https://x.com/Tesla/status/1962591324022153607
Perhaps the thread title could use a small update, as Tesla’s own goal no longer seems to be “The leading car manufacturer of the future” but rather “The leading AI object manufacturer of the future” or something similar
5 Likes
It would be good to update the title, even though the current one has not yet been achieved except in the form of an electric car (or is it one of the leading ones?). A new title could be, for example: “Tesla’s AI Revolution: Cars, Robots, and the Future”
6 Likes