Semi Analysis: “The reality is that there are more dimensions for scaling beyond simply focusing on pre-training, which has been the sole focus of most of the part-time prognosticators. OpenAI’s o1 release has proved the utility and potential of reasoning models, opening a new unexplored dimension for scaling. This is not the only technique, however, that delivers meaningful improvements in model performance as compute is scaled up. Other areas that deliver model improvements with more compute include Synthetic Data Generation, Proximal Policy Optimization (PPO), Functional Verifiers, and other training infrastructure for reasoning. The sands of scaling are still shifting and evolving, and, with it, the entire AI development process has continued to accelerate. Shifting from faulty benchmarks to more challenging ones will enable better measures of progress. In this report we will outline the old pre-training scaling trend as well as the new scaling trends for post-training and inference time. This includes how new methods will push the frontier – and will require even more training time compute scaling then thought before.”
WSJ: ““Long thinking” didn’t make it into the zeitgeist when OpenAI’s ChatGPT first stunned the world two years ago with rapid replies to questions about almost anything. But it has the potential to reduce or eliminate the errors that frequently peppered those responses. The idea is just what it sounds like, at least at the highest level: Long-thinking AI models are designed to take more time to “think over” the results they generate for us. They will be intelligent enough to give us updates on their progress and ask us for feedback along the way. That can mean spending a few more seconds on a problem—or much, much longer, as Huang indicated in another telling remark last June…As the models’ reasoning ability develops, AI is expected to evolve far beyond the current tech that works on our behalf in customer service or automation, or the even more sophisticated agents that are just beginning to appear.”
NYTimes: “Claude, a creation of the artificial intelligence company Anthropic, is not the best-known A.I. chatbot on the market. (That would be OpenAI’s ChatGPT, which has more than 300 million weekly users and a spot in the bookmark bar of every high school student in America.) It’s also not designed to draw users into relationships with lifelike A.I. companions, the way apps like Character.AI and Replika are. But Claude has become the chatbot of choice for a crowd of savvy tech insiders who say it’s helping them with everything from legal advice to health coaching to makeshift therapy sessions. “Some mix of raw intellectual horsepower and willingness to express opinions makes Claude feel much closer to a thing than a tool,” said Aidan McLaughlin, the chief executive of Topology Research, an A.I. start-up. “I, and many other users, find that magical.””
Tae Kim, author of “The Nvidia Way”: “Two important lessons from the book are: mission is the boss and speed of light. Mission is the boss means making the right decision for the company and the customer—not what’s good for your boss’s boss. In corporate America, a huge percentage of time—30 to 50 percent—is spent doing things that don’t help the end customer but instead make your boss look good to their boss. Minimizing that and focusing on the mission, not someone’s bonus, is critical. Speed of light means working with extreme speed and velocity. At most companies, when you do a project, you have KPIs, and you might say, “I did 10 percent better than last time” or “We’re 10 percent better than the competitor.” If you said that at NVIDIA, you’d get dressed down. They don’t care about what you did last time or how you compare to competitors. They care about what’s physically possible. If everything were perfect—no queues, no lag—what’s the absolute limit of physics?”
Walter Lippmann (1937): “Thus it has come about that under gradual collectivism the struggle for power has become ever more intense. As men learn that their fortunes depend increasingly upon their political position, the control of the authority of the state becomes a prize of infinite value.” [via Cafe Hayek]
