Token pricing is the primary cost structure for accessing large language model APIs. Tokens are not identical to words — they're subword units derived from byte-pair encoding (BPE) or similar tokenization algorithms. A useful approximation is 1 token ≈ 0.75 words in English (100 tokens ≈ 75 words), but this varies by language (non-Latin languages and code tokenize differently). Pricing is typically expressed per million tokens and differs between input (prompt) and output (completion) tokens, with output consistently priced higher — 3-5x in most models — because generation requires more compute than inference over provided context.

The economics of training and serving LLMs are dominated by compute costs. Training frontier models (GPT-4 scale) requires thousands of high-end GPUs for months, with estimated costs of $50-$100 million. Inference — serving responses to users — runs on similar hardware and must be amortized across millions of API calls. The dramatic price reductions of 2023-2024 (OpenAI's GPT-4o input price fell from $30/million tokens to $2.50/million over 18 months) reflect both hardware efficiency improvements and competition from open-source alternatives. Meta's LLaMA models, released open-source in 2023, enabled self-hosting that fundamentally changed the commercial market dynamics.

Context window length has emerged as a key pricing and competitive variable. Earlier models (GPT-3.5) had 4,096-token windows; modern models offer 128,000 to 1 million tokens. Longer contexts cost more per API call because attention mechanisms scale quadratically with sequence length (O(n²) complexity in standard transformers). Techniques including sliding window attention, sparse attention, and state-space models (like Mamba) attempt to reduce this scaling cost. For practical applications, context length determines whether you can include a full document in a single API call — critically affecting use cases like contract analysis, code review, and research summarization.

The AI API market is undergoing rapid commoditization. In 2022, OpenAI held near-monopoly position; by 2024, the market included Anthropic (Claude), Google (Gemini), Meta (Llama via cloud providers), Mistral, Cohere, and dozens of specialized providers. This competition has compressed margins and accelerated pricing decreases. Analysts at Sequoia Capital have noted that AI infrastructure is consuming enormous capital — estimated $75B+ in 2024 datacenter investment — while revenue growth, though rapid, may not justify current infrastructure spending. The long-term economics of the AI API market remain uncertain, with pricing subject to further dramatic shifts as open-source models and hardware efficiency continue to improve.