Explained · 45 second read

Your hand is
the button now.

In short: hand tracking uses a camera to follow the position of your fingers many times a second, so a pinch in mid-air works like a tap on a button. Tap below and watch it recognise the gesture.

A hand outline with tracked landmark points, pinching to select a floating target An open hand with dots on the fingertips, knuckles and wrist. Tapping the button below brings the thumb and index finger together in a pinch, which lights up a target box to the right, connected by a dashed line. Select floating in the air

Every dot is a point the camera tracks on your hand. When your thumb and index fingertip get close enough, that counts as a pinch, and whatever you're pointing at gets selected.

Scroll
What the camera actually sees

About 20 points per hand

No gloves, no markers, no controller. Just a camera reading the shape of your hand, over and over.

Fingertips & knuckles Roughly 20 points across each hand, wrist to fingertip.
Many times a second The position updates continuously, so movement stays smooth, not jumpy.
Both hands at once Most systems track two hands independently, so both can act together.
No gloves or markers It's computer vision, not a special sensor. Just a normal front-facing camera.
What it lets you do

Three things hands replace

Once a camera can see your fingers, your hand does what a mouse, touchscreen, or remote used to do.

Tap Touch a button that isn't really there. A tap in the air selects it.
Pinch to grab and move Pinch an object, carry it with your hand, release to let go.
Point to select at a distance A pointing finger casts a ray, so you can select something you can't reach.

Smart glasses have no touchscreen to tap. Hand tracking is why your hands can still be the input, not just voice. See the full picture at the wearable AR pillar page or read what is wearable AR.

Not concept art. Shipped work.

A workbench operated by hand alone

Noodle, a spatial AI workbench on Snap Spectacles operated entirely by hand tracking
Noodle
Hands-only, on Snap Spectacles
Now you know how it works

Designing an interaction with no touchscreen?
We'll help you get the gestures right.

What is AR Wearable AR Smart glasses Touchless UX design
Start a project

6 years directing AR campaigns, official Snap & TikTok partner. Replies within 48 hours.

Prefer to read the long version instead? →

Quick answers

Does hand tracking need me to wear anything?

No. It works from the built-in camera on smart glasses or a phone, reading the shape and position of your hand. No gloves, markers, or extra sensors.

How does hand tracking know I am pinching?

It watches the distance between your thumb tip and your index fingertip. When those two points get close enough, it counts as a pinch, the same way a tap counts as a click.

Why do smart glasses need hand tracking?

Smart glasses have no touchscreen. Hand tracking is the main way you tap, pinch, and point at what you see, usually paired with voice as a backup.

Does hand tracking work in low light or with your hands full?

It gets less reliable in poor light or when your hands are occupied, which is why most experiences pair it with a voice fallback instead of relying on hands alone.