📚

ComfyUI Tutorial

Basic Introduction Install comfyui Text-to-Image Workflow Interface Node Introduction Image Editing Workflow Inpainting Workflow Lora Workflow Workflow Filesystem Trouble Shooting
/Inpainting Workflow

Precision Surgery: A Complete Guide to Inpainting in ComfyUI

You have just spent twenty minutes dialing in the perfect prompt. The composition is flawless, the lighting is cinematic, but there is one glaring issue: your character has six fingers, or there is a bizarre, artifact-ridden coffee cup floating in the background.

You don't want to reroll the entire image and lose the seed you worked so hard to find. You just want to fix that specific area. This is where inpainting comes in.

Doing this in web UIs like Automatic1111 is straightforward but rigid. Doing it in ComfyUI gives you absolute control over the latent space, allowing you to blend masks, inject specific control nets, and dial in the exact denoise values for seamless corrections.

This guide will walk you through building a rock-solid inpainting pipeline from scratch, covering both standard models (like SDXL) and the newer, more complex Flux ecosystem.

1. The Core Logic of Node-Based Inpainting

Before dragging nodes onto the canvas, you need to understand the data flow. Inpainting is essentially Image-to-Image (I2I), but with a barrier.

In a standard I2I workflow, you encode an image into latent space, add global noise, and let the sampler denoise the entire picture. Inpainting adds a Mask. The mask tells the sampler: "Only apply noise and denoise the pixels inside this white area. Ignore the black area."

To do this in ComfyUI, you need three specific components that differ from a basic generation workflow:

  1. An image input with a defined mask.
  2. A specialized VAE encoding node that respects the mask.
  3. Targeted prompting to describe only what should be inside the masked area.

2. Building the Standard Inpainting Pipeline

Let’s build the foundational workflow. Start with a blank canvas.

Step 1: Image and Mask Input

Right-click and add Add Node > image > Load Image. Upload the image you want to fix. Right-click the image directly inside the node and select "Open in MaskEditor". Draw over the area you want to replace. Keep your brush strokes slightly larger than the object itself to give the model room to blend the edges. Click "Save to node".

Step 2: The Checkpoint and Conditioning

Add your Load Checkpoint node. Note: While you can use standard models, using an "inpainting-specific" model (which has additional channels trained specifically for masking) will always yield fewer visible seams.

Add two CLIP Text Encode (Prompt) nodes.

  • Positive Prompt: Describe only what you want to appear in the mask. If you masked a hand, type a perfect hand, five fingers, resting on the table. Do not describe the rest of the image.
  • Negative Prompt: mutated, missing fingers, extra digits, seamless, bad anatomy.

Step 3: VAE Encode (for Inpainting)

This is the most critical step. Do not use the standard VAE Encode node. Search for and add the VAE Encode (for Inpainting) node.

Connect the nodes as follows:

  • IMAGE from your Load Image node -> pixels input.
  • MASK from your Load Image node -> mask input.
  • VAE from your Checkpoint Loader -> vae input.

The grow_mask_by parameter: This expands your hand-drawn mask by a set number of pixels. Set this to 6 or 8. It creates a buffer zone that helps the new generation blend perfectly into the original image without leaving a harsh, visible line.

Step 4: The KSampler Configuration

Add a standard KSampler. Connect your Model, Positive, Negative, and the LATENT output from your VAE Encode (for Inpainting) node.

Dialing in the KSampler for inpainting requires nuance:

  • Steps: 25-30.
  • CFG: 6.0 to 7.0. Keep it standard.
  • Sampler/Scheduler: dpmpp_2m and karras are highly reliable for structure replacement.
  • Denoise: This is your master control.
    • Set it to 0.4 - 0.5 if you just want to slightly alter what is already there (e.g., changing the color of a shirt).
    • Set it to 0.75 - 0.85 if you want to completely replace the object (e.g., turning a coffee cup into a potted plant).
    • Never set it to 1.0 for inpainting, or it will generate completely disconnected noise that ignores the surrounding context.

Step 5: Decoding

Pull the LATENT output from the KSampler into a VAE Decode node, connect your VAE, and output to a Save Image node. Run the prompt.

3. The Flux Inpainting Revolution (Differential Diffusion)

If you are working with the Flux model family, the rules change entirely. Flux does not use traditional inpainting models, nor does it play nicely with the standard VAE Encode (for Inpainting) node right out of the box.

Because of the way Flux handles latent noise across its transformer blocks, forcing a standard mask often results in giant black boxes or deep-fried pixels in the masked area.

To inpaint with Flux, you must use Differential Diffusion.

How to adapt your workflow for Flux:

  1. Standard VAE Encode: Instead of the inpainting-specific VAE node, use the standard VAE Encode node. Connect your image to it.
  2. Set Latent Noise Mask: Add a Set Latent Noise Mask node. Connect the LATENT from your VAE encoder to it, and connect the MASK directly from your Load Image node.
  3. The Secret Sauce: Search for the Differential Diffusion node.
  4. Route the MODEL output from your Flux checkpoint loader through the Differential Diffusion node, and then into your KSampler.

Differential Diffusion mathematically bridges the gap between the masked noise and the clean latent background, allowing base Flux models to perform flawless, seamless inpainting without needing a specialized checkpoint.

4. Advanced Troubleshooting: Beating the "Seams"

Even with a perfect setup, inpainting can sometimes look like a bad Photoshop job. Here is how to fix the most common visual artifacts:

  • The Lighting Doesn't Match: If the new object looks like it's glowing or has the wrong shadow direction, your Denoise is likely too low. The model doesn't have enough freedom to redraw the lighting context. Bump the Denoise up to 0.85.
  • Blurry Output in the Mask: This happens when the masked area is physically too small in pixel dimensions. The model is trying to generate a complex object in a 64x64 pixel box. The solution is to use an Image Crop node before encoding, inpaint on the cropped close-up, and then use an Image Composite node to paste it back into the original high-res image.
  • Color Bleeding: If colors from the original object are bleeding into the new one, increase your grow_mask_by value to 12 or 15 to completely obliterate the original edge data.

5. Structuring for Distribution (The JSON Export)

When you have perfected an inpainting pipeline, you rarely want to build it again from scratch. You also might want to share this exact setup with others. ComfyUI's JSON workflow system is perfect for this, but it requires a bit of cleanup before export.

Before saving your workflow:

  1. Clear User Data: Right-click your Load Image node and ensure there isn't a massive, highly personal 4K image baked into the node data. Replace it with a small, generic placeholder image.
  2. Primitive Nodes: If you are sharing this workflow, extract the Denoise and grow_mask_by parameters into Primitive nodes. Route them to the top of your workspace. This creates a "dashboard" effect, so when someone downloads your JSON, they don't have to hunt through the spaghetti logic to find the controls that actually matter.
  3. Export: Click the "Save" button on the floating menu. The resulting JSON file contains the absolute blueprint of your pipeline, ready to be hosted, shared, or imported into another environment.