The Pulse: a trend of trying to cut back on AI spend within eng departments?
Several tech companies, including OpenAI and Uber, are grappling with high AI spending and lack of clear return on investment (ROI). As a result, many organizations are reassessing their AI budgets and considering cost-cutting measures such as token limits and model routing. Engineers are now experimenting with cheaper models for simpler tasks, leading to an increase in AI efficiency-focused sessions and a new focus on cost savings. To mitigate high AI costs, some companies are introducing measures such as token limits, "dumb" model downgrades, and smart model routing based on use case and prompt. Engineers who contribute to cost savings may be rewarded with promotions and pay rises, making it essential to translate AI cost savings to revenue generation or cost reduction.