• 3 Posts
  • 917 Comments
Joined 2 years ago
cake
Cake day: October 23rd, 2024

help-circle















  • new generation of models makes the cost of inference lower, so that with sufficient customer volume, the companies running the models can make enough profit on inference to make up for the staggering up-front capital expenditures

    cost per quality is definitely going down at a fast rate. LLM providers are in extremely competitive field, where open weight models are at a huge competitive advantage for any quality level (privacy, customizability). The competition is all on 2 month release cycles that essentially throw away the old version/code/weights each time. When Claude pretends its newest model is too powerful for non oligarchs to use, it limits its token reach, and then required contribution margin per token.

    The buisness model flaw is “one day, a winner becomes a monopoly, and AGI self improves the model at low (except for ultra expensive compute) cost.” Monopoly pricing power is very hard/impossible to achieve, because if necessary, foreign governments will subsidize competition to not let a hostile US empire AGI monoplist take hold. Due to corrupt energy oligarchy, it is categorically impossible for US hosted services to ever provide comparative value compared to rational economic energy policies outside of the US. Distillation (Teacher/student RL) means that using another AGI (or leading LLM) will improve models that are behind. There will always be competition on the price/quality curve that prevents even the best/most expensive model from capturing all share. There’s always free tier LLM competition availability as well.

    Finally, there are layers above LLMs. Agentic and swarm and “deterministic program access”/validation front ends to LLMs can add various levels of token burn, but also divert most tokens from the expensive LLMs, and iteratively improve output. There isn’t just a cost/quality curve there is a cost/speed/quality/privacy curve, where non AI coordination tools can improve on the latter curve points independently of leading/expensive LLM/AGI quality.