Stable Diffusion Models Study Guide: Checkpoint
Last article we talked about the concept of models in Stable Diffusion, and went through the basic information of common models. Today we are going to introduce the most important main model, Checkpoint.
Let’s take a look at the Checkpoint model first. Checkpoint model also known as Ckpt model or large model. The reason why it is called Checkpoint is because the model will be archived when it reaches the key position during training. It is somewhat similar to the saving progress when we play games, which is convenient for later calls and rollbacks, such as the official v1.5 The model is adjusted from v1.2.
A common training method for the Checkpoint model is called Dreambooth. This technology was originally developed by Google's team based on its own imagen model. It was later adapted and introduced into the Stable Diffusion model and gradually became widely used. To facilitate everyone's better understanding of the differences between the various models, we have compiled the following schematic diagram for the training process of each model. The following is the process of the Dreambooth training model:
- First, add N steps of noise to the training samples to obtain noise images.
- Then add N-1 steps of noise to the training samples to get the sample calibration chart, which will be slightly clearer than the noise image above.
- Input both the noise image and the text vector generated by keywords into the diffusion model to obtain the model output image.
- Compare the model output image and sample calibration image, and fine-tune the diffusion model based on the difference value until it can correlate the keyword and training sample.
- Through this training method, when entering keywords later, the model will draw a picture similar to the training sample.
Through the above training process, it is not difficult to see that the Dreambooth training model obtains a completely new model by fine-tuning the entire network parameters. Therefore, the Ckpt model can learn a new concept very well, and the effect is very good whether it is used to train characters or painting styles. But the disadvantage is that it is more expensive to train. Normally, training a Ckpt model from the official model through Dreambooth is expected to require tens of thousands of pictures, and the file packages of the model are relatively large (at least at the GB level), which is common Model sizes include 2G, 4G, 7G, etc., which are not flexible enough to use.
It should be noted that the larger the model volume, the better the drawing quality. We sometimes see Ckpt models up to more than 10GB in the model community, but this does not mean that these models are very powerful, because in addition to training through Dreambooth, the Ckpt model can also be obtained through model fusion, but if the author does not optimize the model, then the fused model will be mixed with a large amount of junk data. This data has no effect except taking up valuable hard disk space.
The method of using the Checkpoint model is also very simple. After we download the model file, store it in the \models\Stable-diffusion folder under the Stable Diffusion installation directory. If you add a new model while the WebUI is open, you need to click the refresh button on the right to load it, so that you can select the newly placed model.
