Industries are eagerly capitalizing on Large Language Models (LLMs) to unlock the potential within their vast reserves of under-utilized unstructured data. Given that up to 80% of the worlds data is soon forecast to be unstructured, the drive to harness this wealth for innovation and new product development is immense. There is an ironic paradox here: LLMs, by their very design, output more unstructured text data to manage and keep on top of. That is, until very recently! Earlier this month, OpenAI announced that they now support Structured Outputs in the API with general availability. The ability to distill and transform the creative and diverse unstructured outputs of LLMs into actionable and reliable structured data represents a huge milestone in the world of unstructured data ETL (Extract, Transform and Load). However, theres more to this story than meets the eye. Coincidentally, the day before OpenAIs announcement, a paper was published titled Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models, which offers a compelling counterpoint. They demonstrate that LLMs struggle with reasoning tasks when theyre placed under format restrictions. Additionally, the stricter these format restrictions are, the more their reasoning performance drops, revealing a complex interplay between structuring outputs and model performance. Besides the new structured outputs API from OpenAI, there is also a multitude of existing LLM frameworks that have been developed to tease out structured outputs from conventional LLMs. Under the hood, each of these tools works by using a blend of prompt engineering, output parsing, and error handling techniques. This raises the question: How do these tools compare with OpenAIs structured outputs feature on a task that involves both reasoning and output format restrictions?
