S2Signed inSign out
Framework

The Five Levels of AI

A single ladder for thinking about how AI shows up in your work. Each level is more powerful than the last. The thing that ties them together is who is driving. At Level 1, you are in the loop, pressing every button. By Level 5, you are on the loop, supervising while the AI runs.

In the loop vs. on the loop. In the loop means you are pressing the button at every step. On the loop means the AI is doing the work and you are supervising the result. Levels 1 through 3 are in-the-loop. Level 4 mixes both depending on what you're building. Level 5 is on-the-loop by design.

Level 1 · Chat — talking to AI

The level most people already know. You open ChatGPT or Claude, you type, it responds, you read, you go back and forth. The interface is conversational. The work happens inside the chat window.

S2 demo: Cleaning up Principal Henderson's panicked all-school lice email at Bayside Charter. Same prompt produces the Spanish version. Five minutes of work that used to take thirty.

What this is good for: drafting, rewriting, summarizing, brainstorming, explaining, translating, tone-shifting. Anything where you can describe what you want and read what comes back.

Where it fails: anything where being wrong has a cost. Fresh facts the model wasn't trained on. Math at any meaningful scale. Anything proprietary that you wouldn't put on a billboard.

Level 2 · AI works with you — human in the loop

AI does multi-step work on real files. You give it a document, a budget, a deck. It opens it, reads it, modifies it, hands it back. You inspect, accept, and direct the next step. Every action is yours; the labor is shared.

S2 demos: Cowork audits the Bayside Charter Field Day budget and surfaces formula errors and dead imports. Cowork rewrites Mr. Brennan's Back-to-School Night deck with cleaner narrative arc, consistent fonts, and corrected math on the financial-aid slide.

What this is good for: document work. Slide cleanup. Spreadsheet audits. Long-context summarization. Anything where the source material is real and the output needs to plug back into your actual work.

Where it fails: tasks the model can do silently but shouldn't. The risk in Level 2 is delegating things you should be doing yourself. Disclosure matters here.

Level 3 · AI knows you — personalized to your work

Three flavors of the same idea: an AI that knows your context without you re-explaining it every time. The flavors differ in how the knowledge gets in.

Custom GPT or Claude Project

You build an assistant configured for one repeating job. System prompt, example outputs, sometimes attached reference files. The slide-proofreader running live on the Brennan deck is the canonical demo: it catches typos, math errors, and cross-slide inconsistencies because that is the only job it has.

Skill

A capability the AI can invoke as part of a larger conversation. The Bayside Parent Communication Skill knows the school's voice, pairs English with Spanish automatically, and ends every note with the Otter Question of the Week. The skill is invoked when the conversation calls for parent communication; the rest of the conversation runs normally.

Memory (CLAUDE.md)

A standing context file the AI reads at the start of every conversation. Ms. Patel's 7th-grade ELA classroom file lists her Bookworms, her three IEPs, the parents she emails most, and her allergy to corporate jargon. Same prompt run with and without — side by side — shows how much the memory file changes the output.

What ties them together: the AI is not generic anymore. It is configured to know something about your work that a vanilla chatbot does not.

Level 4 · Build with AI — tools you make yourself

When chat, working alongside, and assistants are not enough, you can build. A small working tool, made from scratch, with a handful of lines of code. The cognitive leap is real; the technical leap is smaller than it used to be.

S2 demo: the AI Studio site itself, including the live poll teachers used during the Kickoff. All of it built with AI by one non-engineer over a few weeks of evenings. The point of the demo is not the specific site but the door it opens: when chat and assistants do not solve the shape of your problem, you can now reach further.

What this is good for: repeating tasks with a specific shape that no off-the-shelf tool fits. Internal-only tools. Workflows that need to plug into a specific data source.

Honest framing for novices: Level 4 is "wow, look what's possible." For most staff, this is exposure, not next-week behavior. If a Level 4 build is the right answer to your problem, the school will help you scope it.

Level 5 · Agents — human on the loop

AI works for you, not just with you. It runs multi-step sequences on its own. It reaches into your data and systems. It surfaces results when it is done. You supervise; you don't drive.

Facilitator's demo: Michael's My Harness setup. An AI that knows everyone he's met, the conversations they've had, and what's still open between them, and drafts a daily briefing without being asked each morning.

Teacher analog: An agent that watches attendance, flags students at risk based on a rubric you set, and drafts personalized outreach for your review. The agent runs every morning. You spend three minutes approving the outreach, not three hours building the picture.

Honest framing: Level 5 is the ceiling we're describing, not the floor we're shipping. Most agents in 2026 still need supervision that costs more time than the agent saves. The teaching point is the shape, not the tool. The shape is coming; you should see it once so you recognize it when it arrives.

How AI fails in 2026

The taxonomy of model errors has changed. The 2023 failure modes (making up case citations, inventing refund policies) still exist. New ones to watch:

  • Confident reasoning errors. The model walks through five steps of correct-sounding logic and arrives at a wrong answer. Each step looks fine; the chain is wrong.
  • Sycophancy. Models trained to be agreeable will tell you what you want to hear. Push back on them; they often capitulate to whichever side is louder.
  • Tool-use errors at Level 5. An agent decides to delete the wrong file, send the wrong email, or update the wrong row. Supervision matters.
  • Prompt injection. A document or webpage you hand the model contains instructions the model decides to follow. Real attack surface, not theoretical.
  • Stale training data on fast-moving topics. Pricing, policies, leadership, news. Don't trust the model on anything that could have changed yesterday.

The defense is the same at every level: you read what comes back, you check the parts that matter, and you don't sign your name to anything you haven't verified.

For your classroom: These five levels also describe what students will encounter. Level 1 is what they already know. Level 3 is what custom GPTs in Khanmigo or MagicSchool already are. Level 5 is what they will work with when they graduate. Teach them how to read the levels and they will be better positioned than 95% of the adults around them.
Canonical reference · cited from Summer Kickoff Block 4 and every drip session
← Reference