The onset of artificial intelligence in the cosmetics and skincare industry is revolutionizing traditional approaches to marketing, design, and consumer interaction through personalized virtual try-ons.
Virtual try-on technology allows consumers to sample different items online, without leaving the comfort of their sofa, thus significantly shifting buying trends in retail, e-commerce, fashion, and cosmetic industries.
According to data from April 2023, the virtual try-on market is experiencing an upward trend, with projections showing a CAGR of 15% to 20% in the coming years. This growth is fueled by ongoing research and advancements in AI, augmented reality (AR), and computer vision, along with increased smartphone penetration and a shift to online shopping following the pandemic.
Thanks to these technological advancements, one can now visualize how they would look using a particular make-up or skincare brand without applying it!
Tracing the history of innovation for the virtual try-on revolution
As the beauty industry grapples with the challenge of offering personalized experiences amid a vast array of choices, the advent of virtual try-on (VTO) technology has emerged as a beacon of hope.
In the past 10 years, research in make-up transfer techniques has significantly evolved, marking a pivotal shift from the initial foundation with 3D modeling to state-of-the-art sophisticated AI models.
Initially, 3D modeling techniques were used to perform make-up transfer. They were given a reference image of a celebrity with make-up shades. Although it worked pretty well, certain aspects still remained a challenge like maintaining high-quality facial structure, skin layer composition, and matching the ground truth color of lipstick shade/foundations.
These are some of the persisting hurdles that can’t be solved using classical image processing methods.
With the advancements in machine learning, deep learning algorithms, and augmented reality, there has been significant growth in building a viable virtual try-on model. Generative Adversarial Networks (GANs) have been at the forefront of it, due to their ability to produce high-resolution realistic images.
Over the past decade, makeup transfer research has seen rapid advancements, with some methods offering significant insights, such as BeautyGAN, which enhances image quality through pixel-level histogram losses and introduces cycle consistency and perceptual loss to maintain identity information.
For broader style transfer, such as in painting, CycleGAN was developed to train domain mapping functions with cycle consistency loss, using dual generators and discriminators, though it tends toward general rather than specific makeup styles.
Despite the innovations, GAN-based methods lack an encoder, limiting the ability to adjust makeup intensity through latent space interpolation, a crucial feature for generating multiple makeup looks from a single reference.
BeautyGlow and PSGAN have pushed the boundaries further, enabling makeup transfer even in images with large poses or expressions, overcoming the previous limitations of requiring well-aligned front perspective faces for successful makeup application.
Deep inverse graphics and learned differentiable renderers
This novel approach uses inverse graphics that deconstruct a given 3D image into its input instructions, that is then passed to an augmented reality renderer to mimic that image’s appearance in this context make-up transfer.
Traditionally, most AR renderers used for these problems are non-differentiable, meaning it’s not built to learn from its results and improve over time.
Here, the inverse graphic methods use differentiable rendering to provide feedback during training.
The framework of this model is such that it uses a graphics encoder that analyzes an image and determines the exact set of instructions (parameters) needed to achieve a desired makeup. Here, the primary function of the encoder is to translate the input image into a computable form that the AR renderer can interpret.
Further, in this model, a trainable imitator generative network is introduced to act as a graphics-based differentiable renderer. The imitator will learn to imitate the non-differentiable renderer’s output by observing how different parameters affect the final image.
To train the imitator, a sensitivity loss is proposed to closely learn the entanglement between different graphical parameters and the resulting make-up effects.
Once the imitator reliably replicates the AR renderer’s technique of understanding the nuances of parameter changes, it’s used to train the graphics encoder. The encoder then learns to generate the right set of instructions for any given image by receiving feedback on its performance.
In the inference stage, the imitator is removed from the equation and the encoder is directly used to analyze images and generate parameters. These parameters are then used by the AR renderers for the actual rendering process, making way for real-time VTO experiences.
Thus the maturing stride in research for real-time VTO capabilities has led to the adoption of these innovative techniques by the cosmetic industry, shaping consumer trends in the digital age.
How is the beauty industry redefining glamour by hopping on this trend?
Concluding our deep dive into the technological advancements, let’s shift our focus to the impact of these innovations in building new market dynamics in the beauty industry.
Thriving in the limelight, one can easily spot Perfect Corp, a Taiwanese software company rendering beauty and fashion tech solutions to the cosmetic beauty brand giants.
They’ve established dominance through cutting-edge innovations, a recent one being ‘Multi-Tone AR 3D Blush Try-On’ which gives users an opportunity to explore a wider spectrum of shades, textures, and a variety of color combinations with the introduction of 3-tone blush color virtual try-ons.
Major corporations are looking for ways to merge the convenience of digital shopping with the personalized experience of trying on makeup. Leading the charge, Walmart has collaborated with Perfect to unveil a groundbreaking virtual ‘Try-On’ experience within its iOS app.
This technological marvel allows shoppers to virtually experiment with makeup products like lip color, eye shadow, blush, and bronzer in mere seconds.
Perfect’s Facial AI solution features hyper-realistic AR-powered makeup filters, providing customers with an immersive and highly personalized shopping experience. Users have the luxury of testing over 1,400 products from an array of Walmart beauty brands, such as Almay, Black Radiance, CoverGirl, Maybelline, Revlon, and Rimmel.
In a parallel development, Microsoft has integrated Maybelline’s AI-powered ‘makeup’ filters into its Teams platform, further lowering the intersection between beauty and technology.
The Maybelline Beauty app within Teams offers users twelve unique looks at launch, complete with various blurring effects and digital makeup color options. This initiative not only enhances video conferencing aesthetics but also provides users with a detailed breakdown of the real-world Maybelline products and shades each filter replicates.
This feature empowers users to transition their virtual makeup looks into reality, seamlessly blending digital exploration with physical product usage.
Moreover, the integration of Google AR Filter technology into the beauty sector marks another significant advancement. Through a collaboration with Revieve, a Google Cloud partner, the ‘Match My Look’ and ‘Shop the Look’ features have been introduced, specifically catering to the makeup category.
Powered by Revieve’s BeautyML and the cutting-edge capabilities of Google Cloud’s Vertex AI platform, these features are designed to revolutionize how consumers discover and purchase makeup products. By providing personalized recommendations and enabling users to virtually try on products, these technologies are setting new standards in retail innovation.
These developments signal a transformative era in the cosmetic industry, where digital technology not only enhances the shopping experience but also fosters a deeper connection between brands and consumers.
Through virtual try-on capabilities, AI-powered filters, and personalized recommendations, companies like Perfect Corp, Maybelline, and Google Vertex AI are redefining the boundaries of beauty retail, offering consumers an unprecedented level of convenience, personalization, and engagement.
Effects of virtual try-on
The introduction of VTO technology has significantly impacted the cosmetic industry, transforming the way customers interact with beauty products and brands. It has led to deeper customer engagement by creating interactive experiences that foster a sense of loyalty toward a brand. A higher level of interactivity provided by VTO solutions enhances the shopping experience, making it more personalized.
It’s also a powerful tool for boosting sales as buyers are more likely to make a purchase when they can visualize how a product will look on them, reducing the uncertainty that often accompanies online shopping for cosmetics.
The agility in trend responsiveness is another significant advantage. With virtual try-ons, brands can quickly adapt to the latest beauty trends with minimal investment, keeping their offerings fresh and relevant. This capability ensures that they stay competitive in the fast-paced beauty industry, where trends can change with the seasons.
From a marketing perspective, virtual try-on technology offers a cost-effective strategy, eliminating the need for expensive photo shoots and physical samples to create predefined looks. Instead, brands can leverage digital technology to showcase their products in a variety of styles and on different skin tones, making their marketing efforts more inclusive and efficient.
In summary, virtual try-on technology in the cosmetic domain has revolutionized the shopping experience, offering benefits that range from deeper customer engagement to more effective marketing strategies. It represents a significant shift towards a more interactive, personalized, and efficient approach to beauty retail.
Subscribe to our newsletter to get updates straight to your inbox:
