GPT-4's output will not be reliable enough to plug directly into downstream programs such as databases or virtual assistants with predictable results.
Accurate ●●●●●
“Its natural language output still won’t be something that one can reliably hook up to downstream programs”
- Said on
- 25 December 2022 · Substack post ahead of GPT-4's release (window inferred: GPT-4 was expected in 2023)
- Source
- What to Expect When You’re Expecting … GPT-4 · Marcus on AI (Substack)
- Should come true
- 2023
- How sure they sounded
- one of "seven dark predictions"
- Categories
- AITechnology
- Importance
- 2 of 5 · Matters to software builders using LLMs.
Outcome
Accurate
GPT-4 got function calling (June 2023) and JSON mode (November 2023), but its output still wasn't reliable. OpenAI later reported that gpt-4-0613 scored under 40% on its complex JSON-schema-following eval, and guaranteed schema adherence only came with Structured Outputs in August 2024, after the window closed. The forecast was about reliability, so it is scored accurate rather than partial.
- https://openai.com/index/introducing-structured-outputs-in-the-api/
- https://pureai.com/articles/2024/08/07/openai-launches-structured-outputs.aspx
History
- 2026-10-17 07:40:31 · created by claude
- 2026-10-17 07:40:31 · categorized by claude
- 2026-10-17 07:41:01 · scored by claude