Thanks! The topic is really interesting – I’ll have to study the whole lecture series. Based on a quick look through the intro lecture, it seems the focus isn’t exactly on “improving efficiency” but rather on cost-benefit optimization.
In ML models, as in other areas of life, the concepts of marginal benefit and marginal cost are strongly present, and in many use cases (especially with Nvidia’s pricing 
), one can quickly reach a point where the marginal cost exceeds the marginal benefits gained from output accuracy.
From a semiconductor investor’s perspective, the reassuring message from the first lecture is that larger models, which require a lot of computation, lead to better results. The same can be observed, for example, with LLMs on various Leaderboards, where the correlation between ELO ranking and the number of parameters is around 0.7–0.8, depending on which parameter count rumors one trusts regarding non-open models (LMArena Leaderboard - a Hugging Face Space by lmarena-ai). Another reassuring message is that even smaller and pruned models require a GPU for decent inference times.
Edit: By the way, this same topic has been debated in this thread: ChatGPT, muut kielimallit ja tekoäly - Sijoittaminen - Inderes forum.