lovecomic

Why AI Comics Keep Changing Your Character's Face — and What Fixes It

A text prompt describes a face lossily, so every panel rolls the dice again. Where character drift comes from, and the reference-sheet workflow that actually holds identity steady.

Aug 28, 2026Lovecomic TeamLovecomic Team
Why AI Comics Keep Changing Your Character's Face — and What Fixes It

If you have generated more than a few panels with an image model, you know the scene. Panel one: your detective has a scar on her left cheek, a grey-streaked coat, a cybernetic eye that glows faint blue. Panel four: the scar has moved or vanished, the coat is black, and the eye is suddenly both eyes. The story didn't change. The character did.

This is the single most common reason AI-made comics feel broken, and it is worth understanding properly — because the fixes that work all come from the same root cause.

Text is a lossy spec for a face

Start from how a text-to-image model actually works. It turns random noise into a picture, guided at every step by your prompt. The prompt acts as a constraint on a distribution of images — it narrows down what gets rendered, but it doesn't name one specific picture. "A 25-year-old detective with a cybernetic eye" matches millions of possible faces. Your prompt picks a neighborhood; it doesn't pick a house.

That has a blunt consequence for comics: each panel is an independent sample. Unless the model is given something more specific than words, panel four is a fresh roll of the dice that happens to land near panel one — not a re-rendering of the same person. The drift isn't the model being sloppy. It's information theory. A sentence doesn't carry enough bits to specify a face: the distance between "a woman with a scar" and your woman with your scar is all the detail you didn't write, and cannot write, in words.

So the question becomes: what carries identity better than a description?

What the research says

This problem is old enough to have a research lineage, and the findings line up cleanly.

In 2022, Google Research framed it precisely in the DreamBooth paper: text-to-image models "lack the ability to mimic the appearance of subjects in a given reference set and synthesize novel renditions of them in different contexts." Their answer was personalization — fine-tune the model on a handful of photos of one subject until it binds that subject to a unique identifier, then generate the subject in new scenes, poses, and lighting. It works, but the cost is a training run per character.

A year later, IP-Adapter from Tencent AI Lab made the same idea cheap. Instead of fine-tuning, it adds a lightweight adapter (about 22 million parameters) with decoupled cross-attention — one channel attends to your text prompt, a second attends to an image prompt. The image itself becomes part of the instruction. No per-character training; you just show the model a picture and say "this person, but in this scene."

That's the load-bearing insight for comics: an exemplar image carries identity far better than adjectives. By 2024–2025 the capability had moved into mainstream editor-class models — Google's Gemini image models (the "Nano Banana" line) put "keep this person, change everything else" into consumer tools, with multi-image reference input as a first-class feature.

What this means inside a comic pipeline

An episode is the worst case for identity: the same characters rendered across four to twelve-plus panels, in new poses, angles, and lighting, every single time. Three ingredients hold it together in practice — and they're exactly what we built into Lovecomic:

1. One canonical reference sheet per character. Before any panels are drawn, each recurring character gets a single reference image that fixes face, hair, outfit, and palette. You can steer it — upload your own reference photo — but the decision is made once, not re-litigated per panel.

2. References travel with every panel. When a panel is generated, the upstream reference images are passed to the model as image inputs alongside the panel description. The model never renders "your detective" from memory; it renders her from the sheet. This is the IP-Adapter move — identity by exemplar — applied at every frame.

3. The written directives stay frozen too. Identity drifts in two ways: the person drifts, and the style drifts around them. Each art style carries fixed drawing directives (line quality, shading, palette) that are injected into every panel prompt, so fixing the character doesn't leave the rendering style to chance. Color or black-and-white is likewise a project-level setting, not a per-panel re-roll.

Around those three, the pipeline adds the safety net: a vision-QA pass scores generated panels and lists visible issues, and — crucially — panels are regenerable individually. When one face drifts, you reroll one panel against the same references, not the whole episode.

A practical checklist

If you're fighting character drift today, in any tool:

  • Decide identity once. Make a reference sheet before the first panel; don't improvise faces as you go.
  • Keep the cast small. Every extra recurring character is another identity the references have to pin down in each frame.
  • Treat wardrobe as identity. Outfit, hair, and palette are what readers (and models) recognize; describe them in fixed terms and never vary them between panels.
  • Fix panels, not episodes. A full reroll re-rolls everything that was already right. Regenerate the one panel that's wrong.
  • If a face still won't hold, regenerate the sheet first. A weak reference sheet poisons every panel downstream of it.

Consistency is not a prompt-engineering trick. It's a workflow: define once, reference everywhere, edit locally. That's the same conclusion the research reached — it just took building a comic pipeline around it to make it routine.

References