A One-Person Japanese Course: An AI Pipeline from Textbook PDF to YouTube
Tags: Japanese · AI Pipeline · Claude Code · Codex · Local Models
Hardware: Mac Studio · RTX 4060 laptop
Channel: YouTube @peifengjp
What I wanted was a Japanese course tailored to me: a fixed cast of characters living in one small town, a question before every explanation, a reason behind every rule, and both Chinese and English versions. So I built it as a pipeline with Claude Code, Codex, a Mac Studio, and an RTX 4060 laptop.
The project is called ひたち日本語 / Hitachi Nihongo. The name comes from one of my favorite places in Japan, Hitachi Seaside Park (国営ひたち海浜公園) in Ibaraki — the hills of blue nemophila in spring and red kochia in autumn, right by the sea. That seaside feeling is also where the course's fictional town comes from.
00 · Why Build It Myself
The reason is simple: I wanted study material shaped around the way I actually learn. Listen to a conversation in context, take a guess, see how the new rule is derived from a form I already know, then start speaking right away. That's the order that works best for me, so every lesson follows it.
There are two versions — Chinese and English — because I wanted to compare what it's like to learn Japanese through different languages. Chinese, Japanese, and English overlap in many ways and differ in many others: Japanese shares kanji with Chinese, yet the readings and usage often diverge; most Japanese loanwords come from English. Comparing the three side by side sometimes makes an expression clearer than explaining Japanese on its own. So the Japanese is written once and shared by both versions, while the explanations are written separately for native Chinese speakers and native English speakers — not translated from one another.
The characters are named after two of my favorite shows, Breaking Bad and Better Call Saul: Saul the teacher, Jesse the student, Walter the neighbor, and Kim the coworker — except that Jesse here is a female student. They all live in a fictional seaside town called 潮見町 (Shiomi-chō). The textbook serves only as a syllabus: I take its topics, can-do goals, and grammar points, but every dialogue, announcement, and example sentence is original.
Making one lesson video like this by hand would take weeks. With the pipeline, my own time on a lesson goes mostly into four decisions: confirming the outline, approving the script, picking the illustrations, and choosing the title. The rest is handled by AI and two machines.
| 12 | lessons scripted |
| 2 | language versions |
| 16–23 min | per video |
| 4 | human decisions per lesson |
01 · The Big Picture
| Step | Stage | What happens | Who |
|---|---|---|---|
| STEP 0 | Read the textbook PDF | Extract topics, can-do goals, and grammar points; separate what only needs to be understood from what needs to be spoken | Claude / Codex |
| STEP 1–4 | Outline + two review rounds | One round from an instructional-design view, one from a learner's view, each with a brand-new reviewer | Claude / Codex |
| GATE | I confirm the outline | The outline must begin with "Status: confirmed (date)" before anything moves forward | Me |
| STEP 5 | Bilingual script | script.toml: the single source of truth for lines, translations, explanations, and on-screen elements | Claude / Codex / Mac |
| STEP 6–7 | Two script review rounds | One for language and content; one for a learner and a native reader who read the "student-view draft" | Claude / Codex |
| GATE | I approve each language | A content hash is recorded; change a single character and the approval is void | Me |
| AUDIO | Voicing + reading check | Qwen3-TTS voices the lines; faster-whisper transcribes them back for a reading comparison | Mac Studio |
| IMAGE | Illustration candidates | ComfyUI + Z-Image-Turbo, 9 images per slot tiled into a contact sheet | 4060 / I pick |
| TIMELINE | Timeline + learning focus | What should be on screen at every moment, computed from lines and pauses | Mac Studio |
| FRAMES | Ukiyo-e style frames | Jinja templates + Playwright screenshots, deduplicated and cached by state | Mac Studio |
| VIDEO | Two full-lesson videos | FFmpeg encoding with credits, end card, metadata, and content hash | Mac Studio |
| PUBLISH | Upload package | Video, thumbnail, subtitles, copy; I pick the title and upload myself | Me / Mac |
Bold rows are gates where I have to make the call. Voicing and image generation can run in parallel with review — they don't depend on review status.
The whole pipeline is a Python CLI called fyl. Each step is a subcommand, and every output lands in a file, so the AI agents and I use exactly the same interface:
uv run fyl check ep12 # validate the script, print content hash and review status
uv run fyl read ep12 --locale en # generate the student-view draft
uv run fyl voice ep12 intro # voicing + timeline
uv run --group verify fyl verify ep12 intro # reading check
uv run fyl images generate ep12 --n 3 # illustration candidates
uv run fyl video ep12 --locale zh # full-lesson video
uv run fyl publish ep12 # YouTube upload package
02 · Machines and Models: Who Does What
On the cloud side, I use only two $20 subscriptions. I deliberately keep cloud models on "writing and judgment," and leave every repetitive, slow, run-it-many-times generation job on local hardware. That way, when I change one line in a lesson and re-voice it, I don't have to think about API costs, and the network doesn't matter.
| Role | Responsible for | What's actually used |
|---|---|---|
| Claude Code / Codex | One role, both author and engineer: read the textbook, write outlines and scripts, draft Chinese and English explanations, write the fyl code, spin up independent reviewers, and run the whole pipeline. Both are $20 subscriptions — whichever has tokens left gets the work; for review, I deliberately switch to a model different from the author | CLAUDE.md and AGENTS.md have identical content, so both agents see the same process, rules, and progress and can hand off at any time; the reviewer is defined in .claude/agents/lesson-reviewer.md |
| Mac Studio | All audio, text processing, rendering, and encoding | Qwen3-TTS-12Hz-1.7B-Base-8bit on MLX; faster-whisper for reading checks; fugashi + UniDic for tokenization and part-of-speech; macOS Vision to detect fake text in illustrations; Playwright Chromium for frame capture; FFmpeg for encoding |
| RTX 4060 laptop | Illustration generation | ComfyUI + Z-Image-Turbo (GGUF Q4_K_M, 9 steps, cfg 1, 1344×768), called remotely from the Mac over Tailscale |
| Me | Judgment only | Confirm outlines, approve scripts per language, pick illustrations from candidates, choose YouTube titles, upload |
The 4060 has only 8GB of VRAM, so the image model is a quantized Z-Image-Turbo that produces an image in 9 steps. The Mac Studio's unified memory, on the other hand, is a great fit for TTS on MLX: a lesson has around 140–150 lines across two languages, plus every quoted Japanese segment, and all of it can be voiced locally at its own pace. Everything is cached by content hash, so changing one line only re-voices that line.
03 · From PDF to Outline
The textbook is Irodori: Japanese for Life in Japan, Elementary 2 (いろどり 生活の日本語 初級2), split into per-lesson PDFs kept locally in source/ and never committed to git. When the agent reads a PDF, it does only three things:
- List the lesson's topics, can-do numbers, and which pages hold the grammar notes;
- Separate forms that only need to be understood (such as the passive in a public announcement) from forms that need to be spoken;
- List the N4–N2 points that come up in the dialogues, as candidates for the "Deep Dive" segment.
Then it writes the outline: three situations, one main focus per situation, spelling out the scene, the characters, what the student should listen for in the opening, the sentences they need to say out loud, and which known form the new one is derived from. Take Lesson 12: the textbook's Lesson 18 is about "wanting to start my own company someday." The outline turned it into Jesse wanting to introduce Shiomi-chō to tourists in Japanese, with three main focuses — volitional form + と思います, dictionary form + ために, and て-form + もらいました — mapping neatly onto "plan → prepare → thank."
Originality is a hard rule. Before handing anything to me, the agent checks every Japanese sentence against the textbook's dialogues, notices, and announcements to make sure nothing matches word for word. Version 2 of Lesson 1 turned up two identical sentences, and they were rewritten.
04 · Different Models, Different Perspectives
This is the most important part of the whole pipeline — and one I only added after getting burned. Version 3 of Lesson 1 made it all the way to video before I watched it myself and found several spots I couldn't follow: it asked what 「もし」 means before ever teaching 「もし」, and the explanation used phrases like "the former / the latter" that only work in written text. The author reads its own work with the full instructional design in mind, so every line feels fine; format checks only check format.
So now every lesson's script goes through several layers of review, each looking at something different.
Layer 1: Rule validation with fyl check
Anything that can be written as a rule is written as a rule and enforced by code: pauses longer than 3 seconds; ellipses or tildes in spoken text; kana appearing outside 「」 in the Chinese explanation; quoted Japanese that's only a single kana; fewer than 3 Chinese characters left before or after a quote (a lone 「了」 at the end of a sentence sounds odd when voiced); a character missing a voice for one of the languages. It also prints the content hash and the review and approval status.
Layer 2: Four independent review rounds
Two rounds for the outline, two for the script, and each round spins up a brand-new reviewer subagent. It can read only the specified files — no access to the author's reasoning, earlier reports, or conversation — and is handed only a student profile: has studied N5, doesn't know terms like "continuative form" or "appearance form," is watching on YouTube, can't scroll back, and has only 3 seconds for each question.
- Outline 1 · Instructional design: Does each situation have exactly one main focus? Does practice drill only that? Can the new form be derived from what's already known?
- Outline 2 · Learner: Walk through everything in order of appearance — is every word and term explained before it's first used? If a structural problem turns up, the outline goes back for a rewrite and another review round.
- Script 1 · Language and content: Is the Japanese natural? Are the furigana and translations correct? Is the JLPT level of each point backed by a source?
- Script 2 · Learner and native reader: Reads only the "student-view draft" produced by
fyl read(exactly what the student sees and hears, in order), and deliberately does not read the authoring rules, so it can't rationalize on their behalf. The English version is read from the perspective of a native English speaker.
The reports are saved verbatim in reviews/, and the authoring agent fills in, item by item, "accepted / rewritten / rejected + reason," along with the post-fix hash. A typical round turns up anywhere from 4 to 20-plus issues per lesson.
Layer 3: Review with a different model
Claude Code and Codex play the same role here and read the same project rules (CLAUDE.md and AGENTS.md). Day to day, which one writes a lesson just depends on which has more tokens left: both are on the $20 plan, so when one runs out, the other picks up. Progress lives in project files, so nothing is lost in the handoff.
When it's time to review, though, I deliberately pick a different model from the author. When the same model writes and reviews, its blind spots tend to be the same: phrasing that felt natural while writing also feels natural while reviewing. A different model often catches what the other can't see. A fresh context solves the "knows the design intent" problem; a different model solves the "same sense of language" problem. The two stack.
Layer 4: A machine listens
After voicing, faster-whisper transcribes the audio back locally and compares by reading, not by kanji, so homophones aren't flagged. That's how it caught the wrong reading for 「夜市」 in Lesson 1. It also found that a standalone quote of 「んですか」 was unstable when voiced — the ん kept getting swallowed — so the explanation was changed to quote the full sentence.
The last layer: me
Once both script review rounds are resolved, I read both student-view drafts, watch the draft video, and then approve each language separately. The approval records a content hash: change even a single character afterward and fyl check fails, the video filename automatically gets a -draft suffix, and it has to be reviewed again before I re-approve. The agent can't approve on its own.
These gates are what let me hand most of the writing to AI with confidence. I don't have to proofread 150 lines word by word; I just need to trust that every layer has been read by someone (or some model) from a position it didn't write from — and the final hash guarantees that what I approved is what goes live.
05 · The Script: One File Owns Every Word
script.toml is the single source of truth. Every word on screen and in the audio comes from it; the renderer never makes up its own sentences or translations. An explanation line looks roughly like this — the Japanese explanation is shared by both versions, while the learner-language commentary is written separately per language (simplified below):
[[situations.lines]]
id = "t042"
speaker = "Saul"
step = "explain"
ja = "「行こう」は、「行きます」の 意向形です。"
zh = "「行こう」是「行きます」的意向形。"
en = "行こう is the volitional form of 行きます."
say = { zh = "中文里没有专门的变形,「行こう」就是“走吧”“我打算去”的那种语气。",
en = "Think of it as the Japanese \"let's go\", but it also works for \"I'm going to\"." }
Each situation follows the same five segments: Dialogue (pose a question first so the student listens for the answer) → Explanation (replay the key sentence, ask, highlight the focus, judge right or wrong) → Practice (easy to hard, ending in role-play) → Deep Dive (JLPT points that actually appeared in this situation) → Summary (have the student say the key sentences first).
After listening to early versions, I gave three pieces of feedback, all of which are now rules: pauses were too long (3 seconds max), there was too much filler (no small talk, no previews, no recaps), and there wasn't enough "why" (every rule has to explain its reason — for example, that たら comes from classical Japanese たらば).
06 · Audio: Four Characters, Three Languages
The voices were designed, auditioned, and selected with VoiceDesign in a separate project. Each of the four characters has reference audio in Japanese, English, and Chinese, all from the same candidate and the same seed, with the pitch difference across the three languages within 2 semitones — so Saul speaking Chinese and Saul speaking Japanese sound like the same person. The references are copied in as-is, with SHA-256 checksums, and never modified.
Voicing runs Qwen3-TTS on MLX on the Mac Studio. A few details make it sound like a teacher giving a lesson rather than a machine reading a script:
- Japanese embedded in a Chinese or English explanation (written inside 「」) is cut out and voiced separately in the teacher's Japanese voice, slowed to 0.8×, padded with silence, and spliced back into the sentence;
- The teacher's own Japanese explanations are slowed to 0.9×, while dialogues and demonstrations stay at natural speed;
- Each line is cached by content hash, and Japanese lines share a single recording across both versions;
- Each line is capped at 30 seconds with up to 4 retries, and voicing automatically produces a timeline with segments, learning points, and start/end times for every chunk.
07 · Images: Modern Characters in Ukiyo-e Backgrounds
Illustrations run on ComfyUI on the 4060 laptop. The style pack is called z-image-ukiyoe: characters are drawn in a modern anime style, while backgrounds use the visual language of shin-hanga — flat color, outlines, indigo gradient skies, bands of mist, and curling water lines. In the first batch I had the characters go "ukiyo-e" too, and their faces and proportions turned into old woodblock prints, which looked off. After that, ukiyo-e stayed in the background only.
Every prompt follows a fixed structure: an opening style paragraph that never changes by a single word; characters as placeholders {jesse}, {walter}, {kim} (their appearance is defined in the style pack so they look the same in every lesson); then location, action, who's holding what, and lighting. The suffix specifically excludes text, seals, and title cartouches, because under ukiyo-e influence the model loves to stamp a little square seal with fake characters in the bottom-right corner.
Each slot gets 9 images, tiled into a contact sheet for me to pick from. macOS Vision flags images that seem to contain text, but it often reads clothespins and railings as letters, so a human still has to look. I've also run into an extra hand and a driver's seat on the wrong side of the car — those get discarded and regenerated.
08 · Video: What Belongs on Screen at Every Moment
The video isn't edited — it's computed. The focus module uses the timeline to decide what appears at each moment: Japanese with furigana and the translation during dialogue, hidden answers during question pauses, a highlight at the reveal, conjugation cards during practice. The ukiyoe module handles the look, written as HTML with Jinja templates and screenshotted by Playwright in Chromium.
Each distinct screen state is captured only once, cached by the hash of its HTML, so a 20-minute video only needs a few hundred screenshots. FFmpeg then encodes the frames and audio into an MP4, embedding author, copyright, URL, and content hash, with a JSON file alongside recording each segment's start and end time. Videos for languages that haven't been approved automatically get a -draft suffix.
After rendering, the agent re-renders every page with the real timeline to check for text overflow, whether answers in practice and summary questions are properly hidden, and whether character names get wrapped mid-word — then fully decodes the audio and video tracks once more.
09 · Publishing
Each lesson's YouTube copy (candidate titles, the opening of the description, tags, thumbnail text, pinned comment) is drafted alongside the script and reviewed in the second script round. Once I pick a title, fyl publish generates an upload package: the final video, thumbnail, subtitles, copy ready to paste, and an upload-settings checklist. The upload itself I do by hand. All finished lessons are on my YouTube channel: youtube.com/@peifengjp.
10 · A Few Takeaways
- Have someone other than the author read it from the student's seat. AI writes fast, but just like people, it fills in the design intent when it reads its own work. A brand-new reviewer with no context is the highest-value link in this whole pipeline.
- If a rule can be code, don't put it in the prompt. Pause length, ellipses, quote length — one
fyl checkis more reliable than ten reminders. - Single source of truth plus content hashes. All text comes from one script, and approval is bound to a hash, so "the version I approved" and "the version that went live" can never drift apart.
- Write rules and progress into project files.
CLAUDE.mdandAGENTS.mdrecord the process, the reason behind every rule, and each lesson's progress, so whether it's Claude or Codex, a new session can pick up right where things left off. Rotating between two $20 subscriptions is enough for one person to build an entire course. - Local models are good enough. One Mac Studio handles voicing and rendering, one 4060 handles images, and cloud models only write and judge. Repetitive generation can be redone freely.
- I keep only the judgment. The direction of an outline, whether a script is approved, which image looks right, what the title should be — those are the calls only I can make, as the student and the channel owner. Everything else can be handed off.
This started as study material just for me. By Lesson 12, I realized the biggest payoff was actually the process of writing outlines and approving scripts: for every lesson, I have to think hard about which form I already know a grammar point grows out of, and where a student is going to get stuck. AI does the heavy lifting, and what's left over is exactly the learning itself.
ひたち日本語 / Hitachi Nihongo · Peifeng Wang · YouTube @peifengjp