Two AI Failures in One Demo: Claude Code Fixes Spelling Instead of Schema Error, OpenAI Mangles Custom Field Mapping

During a workshop at Prismatic, an engineer built a B2B integration end-to-end on stream. Two AI tools failed in distinct ways, illustrating that real-world agent behavior is chaotic and non-deterministic.

Claude Code: Solved the Wrong Problem

Claude Code scaffolded a config wizard using JSON Forms in about 30 seconds. The generated wizard looked fine, but a JSON schema validation error surfaced during testing — something about "must not have fewer than one items." When the engineer asked Claude to fix it, the agent spent the next few minutes fixing spelling warnings in the file instead of addressing the schema error. The engineer eventually said "sure hope it's doing more than fixing spelling issues" and bailed, pasting in code from a dry run done the night before.

OpenAI: Garbage on First Attempt at Weird Fields

The integration calls OpenAI at runtime to generate default field mappings between a customer's Salesforce schema and the destination app. For a normal Salesforce contact (email-to-email, company-to-company), it worked fine — "boring" according to the author. But on a custom record type with deliberately weird field names — Group name, Internet address, Physical place, Internet email address — the first call returned garbage. A second try got it all right.

Key Takeaways

• Boring schemas undersell LLMs — they make agent use look like overkill. The weird, custom cases are where it earns its keep, but most demos avoid those for simplicity.
• Live failures are more useful than successes. Anyone who's worked with agents knows they're chaos. The "fixed spelling instead of schema error" behavior is something no docs would predict.
• Different failure shapes: Claude Code had everything it needed but worked on the wrong problem. OpenAI "knew" the answer but didn't surface it the first time. The shape of failure might indicate how to deploy each tool.

The author works at Prismatic but didn't share a link, focusing on the learning opportunity rather than self-promotion.