TOKYO, Japan — Through facial recognition and augmented reality technology, all it takes is the flick of a digital brush to apply perfectly winged eyeliner or a shimmer of lip gloss to one's face. But what if you could design your own makeup that could be worn in both the physical and digital world?

At this month’s CEATEC, the largest annual IT and electronics tradeshow in Japan, Panasonic introduced its new “Makeup Design Tool” that lets users “freely create digital professional makeup.” But this isn’t another digital tool designed to let e-commerce shoppers virtually try before they buy. Instead, the experience is intended to take place and stay in the digital world. Consumers can virtually design makeup with a touch pen and apply the product to a digital image. With the video simulation mode, makeup can also be applied to moving facial images taken with a video camera.

The tool was designed for beauty industry professionals or to be used as makeup technique training at photo studios, wedding venues or cosmetology schools. “Beauty consultants often refer to designs created on paper or conceptual photographs. But it’s difficult to share one’s vision and sometimes, when the makeup is actually applied to the models or actors, it turns out differently,” explains Yuko Matsuhashi, a spokesperson from Panasonic.

In partnership with Microsoft Japan, Shiseido has introduced a similarly practical offering. The “Telebeauty” app, geared towards women at work, aims to boost the wearer’s confidence while holding conference calls. With four styles of makeup to choose from — natural, feminine, trendy and cool — the software detects the user’s facial features and then applies makeup digitally, giving the impression that the user is wearing makeup when captured on a computer screen. Even if the user moves, the virtual makeup stays in place.

https://youtu.be/dIPaTBv2U0c

According to Shiseido, consumers have been demanding technology that helps them overcome the pressure of looking good during long or remote working hours. In particular, consumers are looking to overcome poor camera quality and lighting that reveals poor skin tone, dark circles and visible appearance of pores. The aim, according to the company, is to “support women who actively pursue their careers remotely, be it from home or otherwise.”

“The problem with online shopping is that there’s a certain commitment you have to make. If you pick a colour that doesn’t work for your face, you’re stuck with it for a while,” says Jude Chao, editor at large at online Korean beauty platform, W2Beauty. “[Beauty apps] can help you break out of your comfort zone and experiment in ways that might feel too risky otherwise. You may even find something you love, which you wouldn’t have dared to try on.”

Traditionally, beauty and skincare products, which rely extensively on highly personalised attributes such as skin tone and facial features, have been difficult to sell online, agreed Hannah Symons, beauty and personal care analyst at Euromonitor International. The global colour cosmetics market was worth $61 billion in 2016 and the total beauty and personal care market was worth $444 billion. “However, globally, only 9 percent of colour cosmetics were sold online,” she reveals.

Selfie-Enhancing Apps

Asia leads the world in selfie-taking, according to a 2014 report by Time magazine, where four of the top 10 "selfiest" cities are in Asia — a find that is perhaps not surprising given that more than half of the world's social media users reside here. Selfie culture has become so dominant in Asia that the obsession over achieving the perfect selfie has culminated in the rise of beautifying image technology.

Founded in 2008 by Xiamen-based entrepreneurs Cai Wensheng and Wu Xinhong, Meitu first gained popularity among millennials and Gen-Z users for allowing them to aggressively retouch their faces in photos.

“What we’ve found is that Millennials are not necessarily less brand loyal. However, they love interactive experiences with brands before making a purchasing decision. They do not like being ‘spammed’ by big brand advertisement, and rather, they trust word of mouth. They also love sharing on social media and seeking advice from their peers, which is why virtual try-on features have done so well,” says Leona Tian, Meitu’s head of global public relations.

Today, the Chinese beauty-enhancing selfie app is installed on over 1.5 billion unique mobile devices and is used by more than 450 million monthly users worldwide, generating an average of 6 billion photos and videos each month.

Millennials love interactive experiences with brands before making a purchasing decision.

In June, Meitu added a feature called Counter, which allows existing Makeup Plus app users to test out new cosmetic products virtually. “In addition to our multinational partners such as Estée Lauder and L’Oréal, we have also partnered with local favourites like Lime Crime in the US and Pony Effect from Korea,” says Meitu’s Tian. The company has signed up over 80 brands worldwide since its launch.

W2Beauty’s Chao sees the phenomenon as an interesting form of advertising. “Beauty apps can be fantastic marketing tools. Brands can virtually push out an entire colour cosmetics shade range to all of its most engaged customers. I’ve also seen multiple skincare apps that offer curated production recommendations within the app,” she says.

Meitu is proving that the platform also drives product sales. According to Tian, when Charlotte Tilbury launched their "Hot Lips" collection, 13 percent of users who tried the lipstick through the app ended up clicking on the "purchase" button and were redirected to Harrods' e-commerce site, where the product is available. "The average click-through rate from the app ranges from 10 percent to 23 percent," she says.

“In addition, we’re helping brands build global awareness. Given our massive user base, Makeup Plus has emerged as a new advertising avenue for brands looking to interact with potential customers worldwide,” she adds, noting that Meitu’s growth isn’t limited to the domestic market. In fact, the app experienced a surge of 480 percent increase of new users outside of China this year.

Virtual to Real Life

Soon, consumers won’t just use tech to wear virtual makeup, but they’ll be able to use tech to create physical makeup for themselves.

Unlike other interactive mirrors, Panasonic’s “Future Mirror” comes with a machine that prints a super thin sheet of makeup containing foundation and concealer. It can be applied the same way you would a temporary tattoo (spray it with water, leave it to settle, then peel off when no longer desired). This also means that no excess make-up is wasted.

The mirror also stores results when it performs a scan, so users can track the progress of their skin regime. “We are planning to roll out the Future Mirror into the market during 2018 to 2019. Our first target will be cosmetic counters and beauty salons, as a B2B solution first, rather than something for home use,” says Matsuhashi.

https://www.youtube.com/watch?v=2-xYgPicHa4

This kind of technology is on the rise in Japan and Korea, where brands are focused on creating beauty spaces that seamlessly integrate online and offline experiences. In August, Shiseido launched a digital counselling mirror at its store inside the recently renovated Ginza Six complex in Tokyo.

Hailed as the first of its kind in the Japanese beauty market, the mirror features a touch-panel system that allows shoppers to scan their faces for a skin check. Users can access their data at home by scanning a specialised QR code with a smart phone, allowing for convenience when making repeat purchases. Following a roll out in Japan, Shiseido plans to upgrade its cosmetics counters around the world.

At the Aritaum flagship store in Seoul, operated by Korean beauty powerhouse Amorepacific, there are several stations aimed at offering a personalised service. “Bio Lab” offers a detailed skin consultation for free. “Beauty mirrors” enable visitors to virtually try on makeup products using a tablet linked to the brand’s mobile app.

The approach is working so far. “APMall, Amorepacific’s official online shopping mall, has recorded an average annual sales growth of 50.4 percent over the past three years. The global beauty market is only at the beginning stage of digitisation, so it’s our priority to continue to find ways to integrate technology with the beauty experience,” Jennifer Rho, senior vice president of strategy at Amorepacific Group, tells BoF.

While there’s potential for beauty apps to be part of the modern consumer’s everyday beauty routine, not everyone is convinced. Some critics argue that AR is more of a branding move than a worthy commerce channel.

“Virtual makeup is still in its early stages and remains a novelty. It’s more about the experience and often the shareable element on social media, rather than a genuine desire to test the makeup,” says Euromonitor’s Symons. “The most pertinent use for these technologies is definitely in the matching of foundation shades, but it still has some way to go in its effectiveness.”

“In the end, nothing beats actually trying something on and knowing for sure whether it’ll work for you. The final stage before buying is still physically trying,” adds Chao. “With that being said, apps can make a great first shopping step and be a lot of fun, too.”

Related Articles:

[ How to Build a Beauty Brand in the Digital AgeOpens in new window ]

[ The Business of BeautyconOpens in new window ]

[ Beauty Brands Inch Away from Traditional AdvertisingOpens in new window ]