How to use negative prompts for Stable Diffusion?

The idea of negative prompts can be traced back to the early days of AI art generation, as far back as the 2010s. Before the rise of modern AI systems like Stable Diffusion and DALL-E, the use of negative prompts was already being explored for fine-tuning and refining the output of image generators at the time.
As AI technology evolved and machine learning models became more sophisticated, the idea of negative prompts became increasingly prevalent. Today, most modern AI image generation systems incorporate negative prompts as a way to control and shape the final output.
Table of Contents
In the explore of AI image generation, you might heard negative prompt when related to Stable Diffusion or DALL-E. Before we started, let's try generate a photo of a man.

He seems very serious, with a solemn handsomeness, but we want him to look younger, maybe without the mustache and wrinkles. Let's try adding "no mustache an wrinkles" into the prompt.

Uh-Oh, something went wrong, the Stable Diffusion seems wrongly interpret the prompt, the mustache stays and the wrinkles seems more. To fix this problem, we should have a walk through the mechanism behind the negative prompt at first.
What is Negative Prompt?
"Negative prompts" is a powerful technique that's used in AI image generators like Stable Diffusion and DALL-E to refine, guide, and improve the generated output. The idea is to use negative prompts to direct the AI towards avoiding unwanted elements. This can range from removing elements such as hands or faces from images to modifying existing elements.
The benefits of using negative prompts in AI image generators are immense. By using negative prompts, you can take control over your image generation process, improving the quality of your output, avoiding unwanted elements, and creating more focused and accurate images.
How does Negative Prompts work?
A fact about prompt is that it is converted into embedding vectors and fed to process U-Net noise prediction.
For sampling, the implementation is conducted in samplers. During the sampling process, the algorithm denoises the image with conditional sampling under guide of text prompt, then the sampler denoises the same image with unconditional sampling. The whole diffusion process will be repeated for the number of steps.
With the negative prompt, the algorithm is helping the diffusion process away from random image, unrecognizable images, and move toward the specified image in the text prompt.
What Can Negative Prompts Do?
The benefits of using negative prompts in AI image generators are immense. By using negative prompts, you can take control over your image generation process, improving the quality of your output, avoiding unwanted elements, and creating more focused and accurate images.
Avoiding unwanted elements
Take the picture of "man" above for instance, we can add "mustache" and "wrinkles" into the negative prompt, then we get the ideal picture of a young handsome man.

Refining the outcome
Another significant usage of negative prompts is to modify the environment surrounding the subject, allowing you to change or alter the image's setting. This way, your subject will be depicted in the same manner as what you described in the original prompt, but the background behind them can be completely altered with negative prompts.
See example of image of "Street in Paris", the picture seems fine, but we want it to be more clear, and no one on the street. Using general negative such as "blurry", "low quality" can improve the image that you create.

There, using the negative prompt can upgrade the quality of you image. When working with high quality pictures, you can use tons of general prompts to benefit your creations.

Controlling the mood and style
Another benefit that using negative prompt is to modify the style of the image. Instead of using keywords "focused" in the positive prompt. For instance, you can use word "blurry" into negative prompt instead of using "sharp" in positive prompt to get a clear image.


Summary
In a sense, proficient use of negative prompts can make your work creation more convenient and bring you a better creative experience. We hope this article bring you better understanding of negative prompts and use it more smoothly.