EN / 中文
In the beauty industry, Virtual Try-Ons (VTOs) and Skin Diagnosis have become handy tools to bring consumers closer to products. VTOs blend Artificial Intelligence (AI) with Augmented Reality (AR) to show consumers how products might look on them. Skin Diagnosis uses data to give personalized suggestions.
The main question we're tackling here is whether AI and AR tools really make a difference. Do they genuinely help consumers find and buy the right products? Do they assist brands in persuading consumers their product is a worthwhile purchase?
Let's explore how some of these technologies have been used and whether they genuinely consider consumer needs. As we needed a real-life test subject, I volunteered as the consumer. I combined my background as a strategist at the marketing agency ReflexGroup, with my passion for beauty as a Gen-Z consumer to provide a unique perspective on the subject.
First, let's set the stage. Virtual try-ons (VTO) kicked off in 2012 with L'Oreal Makeup Genius, an app that lets you experiment with makeup virtually. This tech, initially developed by ImageMetrics, a 3D facial animation company, got even better thanks to folks like Modiface and Perfect Corp, who used computer vision to refine it. Nowadays, it's a go-to tool for big beauty brands and stores looking to boost sales. Most of L'Oreal Group's websites have VTOs up and running. Style My Hair is another nifty app that lets you test out various hair colors. And, don't forget Sephora's Virtual Artist, which lets you try hundreds of products right from their website.
While VTO is all about Augmented Reality (AR), AI plays a crucial role in making it spot on. AI comes in to recognize and track facial features and create realistic looks. It also adds a personal touch by using stored data to give you tailored recommendations.
Seeing that L'Oreal was one of the pioneers in using VTOs, I decided to give the Lancôme VTO a try. The website offers this free feature for a ton of products and shades. I decided to test out a few lipsticks. With just a couple of clicks, I selected a shade, set up my camera, and voilà! The 40€ lipstick was magically on my lips!
The Lancôme VTO is a breeze to use, and it's super quick to set up. The only tiny hiccup was that the lip outline could've been more precise (the good news is that AI can help with that). It did the job well enough to give me a decent idea.
Mascaras were very convincing. I tried three different mascaras and each offered a different look, which was interesting. I noticed a slight exaggeration, especially when it came to the length of my eyelashes. AI could fine-tune this with more details.
Now, onto a little experiment. I thought it would be interesting to compare a product I own in real life with the VTO result. So, I visited L’Oreal Paris’ website.
I gave the L’Oreal Live Try-On a go, which I found to be smooth and user-friendly. The layout is sleek and elegant, and products are neatly sorted into categories, making it easy to explore and virtually test them with just a few clicks. I went with their Telescopic mascara, a beloved item in my makeup bag.
Now, let's take a look at the snapshots below. In these VTO pics, my lashes appear impressively long, beautifully curved, and intensely black. Compare it with the picture of me wearing the actual Telescopic mascara, and you'll notice my lashes aren't quite as full (although I love the result).
In summary, while the VTO result isn't perfectly accurate, it doesn't stray too far from reality. It casts just enough magic to make me consider trying and purchasing the product.
I also gave the Brow Brush a try. It's a fantastic product that keeps your brows locked in place, adds some volume, and gives them a beautiful color that lasts all day. I decided to do the same comparison as before.
On one specific angle, the result looked pretty good – the brow demarcation was on point. However, if I shifted slightly, it seemed a bit off. This is where AI could step in and improve the tracking, making it sharper and more consistent. See below the VTO result on the right is very realistic.
I also tried out their True Match Blush, a staple product. While consumers have praised its ability to deliver a wonderfully natural look, I noticed that some of the VTO versions (left) made it seem too intense and not as subtle as the actual product. It didn't capture the essence of the real product. However, the Apricot shade (right) did stand out as exactly what I typically look for in a blush.
I gave the Colour Riche lipstick a shot in a shade that doesn't typically suit me in real life. Surprisingly, the VTO version appeared very subtle and well-defined.
As for the foundation, any shade I tried looked... surprising. See the photo below, for your pleasure.
As a consumer, I find VTOs to be a valuable tool that enhances the connection with unfamiliar products. However, most of the time, VTO versions provide a less subtle and less appealing representation of the actual product. To address this, there's a need to tone down the harshness of renderings and consider the consumer's skin tone and preferences, such as whether they prefer bold or subtle makeup or a natural look, among others. These are all areas that can be enhanced with AI.
VTOs must achieve a high level of accuracy while striking a delicate balance: avoiding exaggeration while offering a complimentary filter to help consumers envision themselves using the product. Poor lighting conditions can result in unflattering outcomes, that don’t inspire consumers to purchase. In short, VTOs need to reach a level that matches the quality of the product and the reputation of the brand.
After makeup, I hopped onto skincare. On some brands’ websites, consumers can upload a photo, answer a questionnaire, or both to analyze and determine their needs and recommend personalized products. Customized compositions based on data input (through questionnaires) have been used by haircare brand Function of Beauty, skincare brand Prose, and Proven (our former client, whom we helped with rebranding). Skin Scanner examples include Ulta’s Haut AI, Sephora Smart Skin Scan, and Vichy Skin Diagnosis.
I tried out the Vichy Skin Consult AI, uploaded the same picture I used above, and answered a quiz about my skin concerns, age, sleeping habits, environment, etc.
I was given personalized recommendations based on the analysis of my picture and my answers. I found the breakdown very convincing (by concerns, what causes them, and how to avoid them). Nuggets of education are always greatly appreciated. The layout of the minisite is again very elegant and seamless, which makes it a pleasure to use.
One flaw I found though, was that the results were not adapted to my actual age or concerns. For context, I turned 27 this year, drink two liters of water per day, sleep eight hours a night, hydrate my skin, wear SPF daily, eat my veggies, don’t smoke, live in the countryside, and drink little to no alcohol (sugar is my vice). My skin concerns revolve around mild hormonal acne, occasional dullness in the winter, and inconsistent sebum production (with my skin being dry during certain phases of my menstrual cycle and oily during others). So, my skin's needs are those of a young woman navigating the complexities of a second puberty -- lucky me.
Yet the recommendations were mainly centered on wrinkles, fine lines, and firmness. Sure, loss of elasticity will soon be a concern and I’m all for a little bit of prevention. However, I felt that the analysis I received was primarily focused on signs of aging. TThe types of questions and the analysis of my photo did not take into account my current concerns, namely everchanging hormonal flux, cycle breakouts, dryness, etc.
To put it briefly, a person with a cycle has needs for their skin that will vary from week to week. Their skin needs hydration during the first phase, antioxidants in the second, exfoliation in the third, and purification during the fourth. Try exfoliating and purification on week one, you will see breakouts. Over hydrate during week 4 and you will clog the pores of everyone in the room. Moisturizing and SPF are non-negotiables throughout. This is a mere sum up, there are so many subtleties
This is just a brief summary; there are many nuances to consider. I pray for the day a skincare brand will develop a line that follows the cycle. Please call ReflexGroup for branding and launch ;)
All in all, the analysis highlighted elements that are not my concerns while Vichy has all the products that could tackle my issues (the Skin Cycling routine is on their homepage). I'm not yet concerned with aging gracefully, my answers to the quiz reflected that, and I wish the diagnosis had considered that.
I wanted to try another quiz to see what recommendations I would receive based on my input. So I took Prose’s quiz.
Prose is a skincare and hair care brand that received widespread success thanks to its customized formulas and minimalistic products. I found that the questions of the quiz were specific enough to constitute relevant data for an accurate recommendation. I felt my concerns were taken into account by the questions. However, the quiz was a bit long and I was left unsure about some of my answers.
Once again, the questions made me consider my life over the next two months, without considering that my skin's condition varies with my cycle. Lastly, because they only shipped to Canada and the US, I had to provide false information about my location, and this affected my results. Throughout, I appreciated the explanation of the benefits of ingredients and products.
Because the routine was so minimalistic (three products), it would be hard for me to use only that throughout the month. However, formulations looked gentle enough not to cause damage if used for four weeks straight. This would mean that, depending on the moment, I might need an extra boost of hydration when my skin feels super sensitive, or a more heavy-duty cleanser for when my skin feels extra clogged.
However, I would 99% buy these products because my name is on them, the packaging is pretty, and the products are eco-friendly and made with clean ingredients. The fact that the products target some of my recurring skin concerns (dullness, dark circles, etc.) is the cherry on top, but it’s not personalized enough for it to be my main reason.
Personalized recommendations based on AI-processed data input are a game-changing tool to ensure that consumers discover the brand's products that are just right for them, provided that the results are accurate and relevant.
The skincare brands predominantly target women, aka individuals experiencing various life stages, such as menstrual cycles, pregnancy, menopause, and more. Their role is to make sure they take those nuances into account by including these specificities in their AI-powered recommendations. If you found yourself unsettled by my discussion of menstrual cycles and you're employed in this industry, you might want to contemplate a career change, just saying.
Skincare brands are the experts that are supposed to protect our skin and teach us about it. Having more educational content about which phase each product is suitable for would be highly beyond valuable, and this is likely the direction we are heading.
From tools that emerged out of necessity to virtually connect products and consumers during the pandemic, Virtual Try-Ons, Skin Scanners, and quizzes have become common ways for makeup and skincare brands to enhance the consumer experience and effectively tap into consumer psychology to address their needs.
My impression is that VTOs currently work well as an addition to the in-store experience, where you can touch, smell, and see the product packaging, but can’t try it on your face for hygiene reasons.
Regarding Skin Scanners, their potential evolution to include more specificity and nuance, especially related to hormonal balance, could bring exciting changes.
AI will undoubtedly enhance the accuracy of these tools as more data is processed and utilized. Incorporating more detailed questions (e.g., lash length, contouring preferences) and additional photo uploads (with or without blush, from different angles) would provide a more realistic representation of the product and generate highly accurate recommendations. Since accuracy is a primary goal and the core reason for using AI in the beauty context, it's definitely worth exploring.
If you'd like to read more pieces like this, I'd be happy to start a GoFundMe campaign for a shopping spree at Sephora to compare Virtual Try-Ons with reality ;)
