Development of Semantic Editing Methods Based on Generative Models

Semantic image editing using generative models is a rapidly developing field, and many approaches exist to solve this problem. One method that utilizes the generative capabilities of StyleGAN [7] and the multimodal latent space of CLIP [16] is DeltaEdit [13], a method that does not directly train on textual prompts. Despite having many advantages over other StyleGAN+CLIP approaches, DeltaEdit performs poorly with complex prompts that correspond to simultaneous changes in multiple image attributes. To address this issue, we propose two modifications to DeltaEdit that have not been previously mentioned in the literature – one uses a style mixing technique (SM-DeltaEdit), while the other involves optimizing a modified CLIP-NCE loss function [21] (CF-DeltaEdit). Through deep quantitative and qualitative analysis, we will demonstrate the superiority of the CF-DeltaEdit method over its predecessor, as well as theoretically highlight the shortcomings of the SM-DeltaEdit method. Code is available at https://github.com/hzchet/semantic-editing.