LLM endpoints are slower and less deterministic than classic CRUD APIs. Design for streaming, partial failure, and clear versioning from day one.
Stream early
Users tolerate latency better when tokens arrive progressively.
Version prompts like code
Treat prompt and model changes as deployable artifacts with rollback.