Estée Lauder Shopping Assistant
An inclusive in-store beauty shopping experience for customers with visual and mobility impairments, through AR navigation, Virtual Try On, and features.
Timeframe
February - March 2023
Skills
User Research
UI/UX Design
Prototyping
Team
Iñigo Zepeda
Lolita Mileta
My Role at ELSA
At Estée Lauder’s Hackathon I took the role of an End-to-End Designer covering all stages all stages of the development from ideation, research, visual styling, interactions and prototyping.
EL hosted the Accessible Beauty Hackathon focused on creating an inclusive technology solutions for beauty consumers. We ideated an app to improve the shopping experience for people with visual and mobility impairments.
How to enhance accessibility in the consumers' shopping experience?
In-person shopping experiences are challenging for people with disabilities. The physical and mental strain of navigating a store without being able to independently grab products can be overwhelming. Current makeup shopping experiences create barriers to true independence and personal expression for people with mobility and visual impairments.
For this hackathon, we focused on two groups: those with physical and visual disabilities. Due to time constraints, the research method was through user insights from existing accessibility focused apps, retail accessibility reports and case studies of indoor navigation solutions.
After analyzing these pain points, we defined features we wanted the app to have. However, we had to do research in order to design them in a user centric approach.
Competitive Analysis
We started by analyzing apps which had features and behaviors that we could potentially implement in ELSA (Estée Lauder Shopping Assistant). Interestingly enough, there is not any app which combines product discovery, virtual try on and shopping assistance while prioritizing accessibility for its users with visual and mobility impairments. This meant that there are no direct competitors, resulting in a big of opportunity to innovate in this sector.
The following graphic shows the researched features, the main description of how they work, and the products that own these features.
Indoor Navigation Apps
Description:
Digital mapping systems that help users navigate indoor spaces through instructions on their phone.
Accessibility Features:
Audio directions designed to guide user to specific locations.
Pain Points:
Navigation can sometimes be not 100% accurate with the product location.
Examples: GoodMaps, GoodMaps Walmart
Virtual Try-On Tools
Description:
AR technology that allows users to preview makeup products on their face in real-time using their device's camera.
Accessibility Features:
Mobility impaired users can try products virtually.
Pain Points:
Lacks audio feedback and is primarily designed for online shopping.
Examples: Sephora Virtual Artist, L'Oréal Genius, ModiFace
AI Assistant Technology
Description:
Chatbot powered by AI that answers questions, provides recommendations, and offers guidance.
Accessibility Features:
Audio input and output capabilities.
Pain Points:
Users may not understand its purpose without guidance and recommendations.
Examples: Seeing AI, Be My Eyes, Aira, Envision
Shopping Carts
Description:
Digital wishlist and cart systems that let users save products and manage their shopping selections.
Accessibility Features:
Limited. Basic screen reader and selection gestures of the phone.
Pain Points:
Optimized mainly for online shopping experiences.
Examples: Amazon, eBay, Etsy, MercadoLibre
User Proto Personas
Based on the information researched, we created user proto personas. From our research, we synthesized two user personas representing two user segments, mobility impaired and visually impaired people.
Patricia, 37
Marketing Manager, lives in Mexico City
About Patricia
Patricia is a marketing manager who loves fashion and beauty. She uses a wheelchair due to a spinal cord injury. She enjoys shopping but finds in-store experiences exhausting and prefers solutions that don't require constant staff assistance.
Goals & Needs
- Navigate stores independently without needing constant employee assistance.
- Access products on high or low shelves that are beyond her reach.
- Collect desired items without physically retrieving them herself.
- Try products virtually before purchasing them in-store.
Frustrations
- Store layouts are confusing with narrow aisles and poor signage.
- Products on unreachable shelves make shopping feel impossible alone.
- Waiting for staff assistance makes her feel dependent and slows shopping.
- Physical exhaustion from navigating large stores limits shopping time.
Zoe, 29
Teacher, lives in Mexico City
About Zoe
Zoe is an English teacher who has been legally blind since birth. She relies on her iPhone's VoiceOver and white cane for navigation. She loves makeup and skincare but struggles to shop independently due to inaccessible product labeling and store navigation.
Goals & Needs
- Navigate store layouts independently using audio guidance.
- Identify products through accessible descriptions instead of visual packaging.
- Try makeup shades virtually to understand colors without physical testing.
- Shop confidently without relying on family, friends, or strangers.
Frustrations
- Unable to navigate unfamiliar stores without constant guidance from others.
- Product packaging lacks tactile or audio information for identification.
- Virtual try-on features aren't screen reader compatible or lack descriptions.
- Dependence on others for shopping undermines her autonomy and confidence.
Main Features
We defined the core features that ELSA would need to in order to provide an accessible and comprehensive in-store shopping experience.
Browse & Find Products
Virtual Try On
AI Assistant
Wishlist
User Journey
Mapping the complete experience from store entry to purchase, identifying key touchpoints where accessibility features would have the most impact on user independence and satisfaction.
User Journey Map
Sitemap
App Sitemap
User Flow
This user flow shows the product search flow, from query input (voice or text) to the product selection.
User Flow - Search
Wireframes
Wireframe of the main pages in the app from onboarding to product selection.
Onboarding
First time users are welcomed with an onboarding flow that introduces the app's key features.
Onboarding Interaction
Finding a Product
There are two ways to find a product. The first is browsing through categories from the main page until locating the desired product (left video). The second is using the search function, which displays recommendations as the user types (right video).
Find Through Category Browsing
Find Through Search Function
Find in Store
The AR powered navigation tool assists users with audio and visual guidance to help them locate products within the store independently.
Virtual Try On
The accessible Virtual Try On tool allows users to preview makeup products using AR technology, providing users who might not reach some products, the availability to see them on.
Wishlist
Designing a wishlist feature that enables users to save desired products and share them via QR code with store staff for assistance in collecting items, promoting a much more fluid buying experience for the user.
AI Chatbot
Implementing an AI-powered assistant that provides product recommendations, answers questions, and offers guidance throughout the shopping experience with full voice interaction support.
Style Guide & Components
Branding elements including colors, fonts, and typography was taken from the official style guide of Estée Lauder. The components and its interaction are documented to establish an efficient developer handoff.
Project Impact
Our hackathon submission amplified awareness of accessibility barriers in the beauty shopping industry and demonstrated a viable path toward inclusive shopping experiences, similar to existing solutions like GoodMaps, which has revolutionized indoor navigation for major retailers including Walmart Mexico.
The broader impact lies in the potential for real-world implementation. If deployed in Estée Lauder stores, ELSA would empower customers with mobility and visual impairments to shop independently. The hackathon's recognition validates both the feasibility and market need for accessible technology in the beauty industry.
Future Work
The next step would be conducting usability testing with actual users who have mobility and visual impairments to validate our design decisions and identify areas for improvement. Real user feedback is essential to ensure the app truly meets accessibility needs.
Additionally, developing a companion app for store employees would complete the ecosystem, making it easier for staff to assist customers and manage wishlist requests efficiently. Expanding accessibility features to support other disabilities, such as hearing impairments, would further our goal of creating an inclusive shopping experience for everyone.
Learnings and Reflections
We are incredibly grateful for the opportunity to ideate a solution for people with mobility disabilities who want to enjoy the beauty shopping experience at Estée Lauder. Throughout the creation process, we gained valuable insights into the unique challenges faced by people with disabilities and were able to design an application that directly addresses their needs.
Overall, creating this project reminded us the importance of inclusive design and the potential that technology has in order to impact people’s lives in a positive way. We hope that ELSA will be implemented in the future, and that it serves as an inspiration for other people and companies to prioritize accessibility in their products and services.