For the previous decade and a half, I’ve been exploring the intersection of expertise, schooling, and design as a professor of cognitive science and design at UC San Diego. Some of you may need learn my current piece for O’Reilly Radar the place I detailed my journey including AI chat capabilities to Python Tutor, the free visualization device that’s helped tens of millions of programming college students perceive how code executes. That expertise acquired me fascinated about my evolving relationship with generative AI as each a device and a collaborator.
I’ve been intrigued by this rising apply referred to as “vibe coding,” a time period coined by Andrej Karpathy that’s been making waves in tech circles. Simon Willison describes it completely: “When I talk about vibe coding I mean building software with an LLM without reviewing the code it writes.” The idea is each liberating and barely terrifying—you describe what you want, the AI generates the code, and you merely run it with out scrutinizing every line, trusting the general “vibe” of what’s been created.
Learn sooner. Dig deeper. See farther.
My relationship with this method has developed significantly. In my early days of utilizing AI coding assistants, I used to be that one that meticulously reviewed each single line, usually rewriting vital parts. But as these instruments have improved, I’ve discovered myself regularly letting go of the steering wheel in sure contexts. Yet I couldn’t totally embrace the pure “vibe coding” philosophy; the professor in me wanted some high quality assurance. This led me to develop what I’ve come to name “vibe checks”—strategic verification factors that present confidence with out reverting to line-by-line code critiques. It’s a center path that’s labored surprisingly properly for my private initiatives, and at the moment I need to share some insights from that journey.
Vibe Coding in Practice: Converting 250 HTML Files to Markdown
I’ve discovered myself more and more turning to vibe coding for these one-off scripts that resolve particular issues in my workflow. These are sometimes duties the place explaining my intent is definitely simpler than writing the code myself, particularly for information processing or file manipulation jobs the place I can simply confirm the outcomes.
Let me stroll you thru a current instance that completely illustrates this method. For a category I train, I had college students submit responses to a survey utilizing a proprietary net app that supplied an HTML export possibility. This left me with 250 HTML recordsdata containing precious pupil suggestions, but it surely was buried in a large number of pointless markup and styling code. What I actually wished was clear Markdown variations that preserved simply the textual content content material, part headers, and—critically—any hyperlinks college students had included of their responses.
Rather than penning this conversion script myself, I turned to Claude with an easy request: “Write me a Python script that converts these HTML files to Markdown, preserving text, basic formatting, and hyperlinks.” Claude recommended utilizing the LovelySoup library (a strong alternative) and generated a whole script that may course of all recordsdata in a listing, making a corresponding Markdown file for every HTML supply.
(In retrospect, I spotted I most likely may have used Pandoc for this conversion process. But within the spirit of vibe coding, I simply went with Claude’s suggestion with out overthinking it. Part of the attraction of vibe coding is bypassing that analysis section the place you examine totally different approaches—you simply describe what you need and roll with what you get.)
True to the vibe coding philosophy, I didn’t assessment the generated code line by line. I merely saved it as a Python file, ran it on my listing of 250 HTML recordsdata, and waited to see what occurred. This “run and see” method is what makes vibe coding each liberating and barely nerve-wracking—you’re trusting the AI’s interpretation of your wants with out verifying the implementation particulars.
Trust and Risk in Vibe Coding: Running Unreviewed Code
The second I hit “run” on that vibe-coded script, I spotted one thing which may make many builders cringe: I used to be executing fully unreviewed code on my precise pc with actual information. In conventional software program growth, this is able to be thought of reckless at greatest. But the dynamics of belief really feel totally different with trendy AI instruments like Claude 3.7 Sonnet, which has constructed up a status for producing moderately secure and useful code.
My rationalization was partly based mostly on the script’s restricted scope. It was simply studying HTML recordsdata and creating new Markdown recordsdata alongside them—not deleting, modifying current recordsdata, or sending information over the community. Of course, that’s assuming the code did precisely what I requested and nothing extra! I had no ensures that it didn’t embrace some surprising conduct since I hadn’t checked out a single line.
This highlights a belief relationship that’s evolving between builders and AI coding instruments. I’m way more keen to vibe code with Claude or ChatGPT than I’d be with an unknown AI device from some obscure web site. These established instruments have reputations to take care of, and their mum or dad firms have sturdy incentives to stop their techniques from producing malicious code.
That mentioned, I’d like to see working techniques develop a “restricted execution mode” particularly designed for vibe coding situations. Imagine having the ability to specify: “Run this Python script, but only allow it to CREATE new files in this specific directory, prevent it from overwriting existing files, and block internet access.” This light-weight sandboxing would supply peace of thoughts with out sacrificing comfort. (I point out solely limiting writes relatively than reads as a result of Python scripts sometimes must learn numerous system recordsdata from throughout the filesystem, making learn restrictions impractical.)
Why not simply use VMs, containers, or cloud providers? Because for personal-scale initiatives, the comfort of working straight alone machine is tough to beat. Setting up Docker or importing 250 HTML recordsdata to some cloud service introduces friction that defeats the aim of fast, handy vibe coding. What I would like is to take care of that comfort whereas including simply sufficient security guardrails.
Vibe Checks: Simple Scripts to Verify AI-Generated Code
OK now come the “vibe checks.” As I discussed earlier, the good factor about these private information processing duties is that I can usually get a way of whether or not the script did what I meant simply by analyzing the output. For my HTML-to-Markdown conversion, I may open up a number of of the ensuing Markdown recordsdata and see in the event that they contained the survey responses I anticipated. This guide spot-checking works moderately properly for 250 recordsdata, however what about 2,500 or 25,000? At that scale, I’d want one thing extra systematic.
This is the place vibe checks come into play. A vibe test is basically an easier script that verifies a primary property of the output out of your vibe-coded script. The key right here is that it needs to be a lot less complicated than the unique process, making it simpler to confirm its correctness.
For my HTML-to-Markdown conversion venture, I spotted I may use an easy precept: Markdown recordsdata needs to be smaller than their HTML counterparts since we’re stripping away all of the tags. But if a Markdown file is dramatically smaller—say, lower than 40% of the unique HTML dimension—which may point out incomplete processing or content material loss.
So I went again to Claude and vibe coded a test script. This script merely:
- Found all corresponding HTML/Markdown file pairs
- Calculated the scale ratio for every pair
- Flagged any Markdown file smaller than 40% of its HTML supply
And lo and behold, the vibe test caught a number of recordsdata the place the conversion was incomplete! The unique script had did not correctly extract content material from sure HTML constructions. I took these problematic recordsdata, went again to Claude, and had it refine the unique conversion script to deal with these edge circumstances.
After just a few iterations of this suggestions loop—convert, test, determine points, refine—I finally reached some extent the place there have been no extra suspiciously small Markdown recordsdata (properly, there have been nonetheless just a few under 40%, however guide inspection confirmed these had been right conversions of HTML recordsdata with unusually excessive markup-to-content ratios).
Now you may moderately ask: “If you’re vibe coding the vibe check script too, how do you know that script is correct?” Would you want a vibe test to your vibe test? And then a vibe test for that test? Well, fortunately, this recursive nightmare has a sensible answer. The vibe test script is usually an order of magnitude less complicated than the unique process—in my case, simply evaluating file sizes relatively than parsing complicated HTML. This simplicity made it possible for me to manually assessment and confirm the vibe test code, even whereas avoiding reviewing the extra complicated unique script.
Of course, my file dimension ratio test isn’t good. It can’t inform me if the content material was transformed with the correct formatting or if all hyperlinks had been preserved accurately. But it gave me an affordable confidence that no main content material was lacking, which was my major concern.
Vibe Coding + Vibe Checking: A Pragmatic Middle Ground
The take-home message right here is easy however highly effective: When you’re vibe coding, all the time construct in vibe checks. Ask your self: “What simpler script could verify the correctness of my main vibe-coded solution?” Even an imperfect verification mechanism dramatically will increase your confidence in outcomes from code you by no means truly reviewed.
This method strikes a pleasant steadiness between the pace and inventive circulate of pure vibe coding and the reliability of extra rigorous software program growth methodologies. Think of vibe checks as light-weight assessments—not the great check suites you’d write for manufacturing code, however sufficient verification to catch apparent failures with out disrupting your momentum.
What excites me concerning the future is the potential for AI coding instruments to counsel acceptable vibe checks routinely. Imagine if Claude or related instruments couldn’t solely generate your requested script but in addition proactively provide: “Here’s a simple verification script you might want to run afterward to ensure everything worked as expected.” I believe if I had particularly requested for this, Claude may have recommended the file dimension comparability test, however having this constructed into the system’s default conduct could be extremely precious. I can envision specialised AI coding assistants that function in a semi-autonomous mode—writing code, producing acceptable checks, working these checks, and involving you solely when human verification is really wanted.
Combine this with the sort of sandboxed execution setting I discussed earlier, and you’d have a vibe coding expertise that’s each releasing and reliable—highly effective sufficient for actual work however with guardrails that forestall catastrophic errors.
And now for the meta twist: This whole weblog put up was itself the product of “vibe blogging.” At the beginning of our collaboration, I uploaded my earlier O’Reilly article,”Using Generative AI to Build Generative AI” as a reference doc. This gave Claude the chance to investigate my writing type, tone, and typical construction—very similar to how a human collaborator may learn my earlier work earlier than serving to me write one thing new.
Instead of writing the complete put up in a single go, I broke it down into sections and supplied Claude with a top level view for every part separately. For each part, I included key factors I wished to cowl and typically particular phrasings or ideas to incorporate. Claude then expanded these outlines into totally fashioned sections written in my voice. After every part was drafted, I reviewed it—my very own model of a “vibe check”—offering suggestions and requesting revisions till it matched what I wished to say and how I wished to say it.
This iterative, section-by-section method mirrors the vibe coding methodology I’ve mentioned all through this put up. I didn’t want to write down each sentence myself, however I maintained management over the course, messaging, and last approval. The AI dealt with the execution particulars based mostly on my high-level steering, and I carried out verification checks at strategic factors relatively than micromanaging each phrase.
What’s significantly attention-grabbing is how this course of demonstrates the identical ideas of belief, verification, and iteration that I advocated for in vibe coding. I trusted Claude to generate content material in my type based mostly on my outlines, however I verified every part earlier than transferring to the following. When one thing didn’t fairly match my intent or tone, we iterated till it did. This balanced method—leveraging AI capabilities whereas sustaining human oversight—appears to be the candy spot for collaborative creation, whether or not you’re producing code or content material.
Epilogue: Behind the Scenes with Claude
[Claude speaking]
Looking again at our vibe running a blog experiment, I ought to acknowledge that Philip famous the ultimate product doesn’t totally seize his genuine voice, regardless of having his O’Reilly article as a reference. But in line with the vibe philosophy itself, he selected to not make investments extreme time in countless refinements—accepting good-enough relatively than good.
Working section-by-section with out seeing the complete construction upfront created challenges, much like portray elements of a mural with out seeing the whole design. I initially fell into the entice of copying his define verbatim relatively than reworking it correctly.
This collaboration highlights each the utility and limitations of AI-assisted content material creation. I can approximate writing types and broaden outlines however nonetheless lack the lived expertise that provides human writing its genuine voice. The greatest outcomes got here when Philip supplied clear course and suggestions.
The meta-example completely illustrates the core thesis: Generative AI works greatest when paired with human steering, discovering the proper steadiness between automation and oversight. “Vibe blogging” has worth for drafts and outlines, however like “vibe coding,” some type of human verification stays important to make sure the ultimate product actually represents what you need to say.
[Philip speaking so that humans get the final word…for now]
OK, that is the one half that I wrote by hand: My parting thought when studying over this put up is that I’m not pleased with the writing high quality (sorry Claude!), but when it weren’t for an AI device like Claude, I’d not have written it within the first place on account of lack of time and vitality. I had sufficient vitality at the moment to stipulate some tough concepts, then let Claude do the “vibe blogging” for me, however not sufficient to totally write, edit, and fret over the wording of a full 2,500-word weblog put up all on my own. Thus, identical to with vibe coding, one of many nice joys of “vibe-ing” is that it drastically lowers the activation vitality of getting began on inventive personal-scale prototypes and tinkering-style initiatives. To me, that’s fairly inspiring.