AI Gesture Recognizer

What it recognises

Upload a photo of a hand making a clear gesture and the page names it: thumbs up, thumbs down, victory, pointing up, open palm, closed fist, I love you, or nothing at all. It shows an emoji, the label, a confidence percentage and a progress bar, and marks the 21 hand joints on the photo.

The vocabulary is fixed. This is not a general gesture interpreter and you cannot teach it a new sign. It recognises the small set of static hand shapes the bundled model was trained on, and anything outside that set comes back as "None".

The model

The eight categories are the canned set that ships with the model. The page's emoji table also contains entries for OK and Peace, but the model does not output those category names, so in practice you will never see them. Treat the list above as the complete vocabulary.

The model reads a single hand. In a photo with two hands, it returns the first one it finds and ignores the other, so a two-handed sign will be judged on whichever hand the detector picked.

What appears on screen

The selected photo is drawn onto a canvas at its natural size. Each of the 21 joints is marked with a white dot and a blue ring, giving a light constellation over the hand. No lines connect the joints, so the shape is visible through the dots rather than traced as a skeleton.

The side panel shows a large emoji, the label with underscores replaced by spaces, and the confidence to one decimal place, for example "97.4%". A slim progress bar repeats the same number visually. Nothing is drawn on the image itself apart from the dots.

If no hand is found, or the hand is present but the shape matches nothing, the panel shows a short message and the status line reports that no gesture was detected. There is no download button at any point — the result is for reading on screen.

Step by step

1. Click the upload area and choose a photo. Drag and drop is not supported.
2. Wait for the runtime and the gesture model to download on first use. The panel names each stage.
3. Recognition runs automatically once the image is decoded.
4. Read the label and the percentage. The dots show which hand the model actually used.
5. Upload another photo to try again; the loaded model makes the second run fast.

The Load AI Model button preloads the model. When it finishes it reports "Model Ready" and disables itself.

Getting a reliable result

• Show the palm or the back of the hand clearly. The model needs the whole handshape, not a sliver.
• Fill a good part of the frame. A hand at the edge of a wide shot gives too few pixels.
• Keep the fingers separated. Overlapping fingers blur the distinction between a fist and a loose curl.
• Use even light. Strong shadows across the palm hide the finger positions.
• Avoid motion blur. A still frame from a video often works better than a snapshot taken mid-move.

The confidence number is useful for comparing attempts on the same image, but it is not a probability that the label is correct. A 99 percent score on a blurry hand can still be wrong, and a 60 percent score on a clear one can be right.

Where it is useful

• Prototyping a touchless interface. Check whether a camera angle and lighting setup produce stable labels before building the live version.
• Teaching. A concrete demonstration of how a classifier maps hand shapes to categories.
• Accessibility demos. Show how a small set of gestures can trigger simple actions.
• Testing a dataset. Confirm that photos labelled "thumbs up" actually contain a recognisable thumbs up.
• Quick visual checks. Verify that a photographed hand pose reads the way you intended.

Where it fails

Static, unusual or asymmetric hand shapes are the weak spot. A shaka, a pinch, a crossed-finger gesture or a partially closed hand usually lands on "None" or on the nearest category with a low score. Cultural gestures outside the eight are not represented at all.

Handedness matters: a thumbs up made with the left hand still reads as thumbs up, because the class is about shape rather than side. But a gesture viewed from behind the hand, or from a steep angle, can change the apparent finger positions enough to flip the label.

Occlusion hurts. A hand holding an object, wearing a thick glove or partly out of frame loses the cues the model needs. So do very small hands, low resolution, heavy shadow and busy backgrounds that confuse the initial hand detector.

Two-handed signs are effectively unsupported, since only one hand is read. Dynamic gestures — a wave, a sequence, a sign that changes over time — cannot be recognised from a single still image at all.

What it does not do

No custom gestures, no training, no multi-hand recognition, no sequence or motion recognition, no video, no webcam, no download, no coordinate export and no confidence threshold control. It does not translate sign language, does not read finger spelling and does not identify the person.

Privacy

The photo is processed in the browser. It is decoded locally, passed to the WebAssembly model and drawn to a canvas; nothing is uploaded and nothing is retained. The runtime and weights are fetched from public CDNs on first load, which shows the tool was opened but not what was photographed.

Limits

One image, one hand, one of eight labels. No batch, no video, no download, no emoji customisation and no way to see the full probability list — only the top category and its score. Large images take longer to decode, and a modern browser with WebAssembly plus network access is required.

Common mistakes

• Expecting any gesture to work. Only the eight built-in categories are recognised.
• Uploading two hands. Only one is read, which breaks two-handed signs.
• Reading a low score as a near miss. It often means the shape is simply outside the vocabulary.
• Assuming "None" means no hand. It can mean a hand was found but matched no category.
• Looking for a download. There is no export on this page.
• Using a dark, backlit photo. Silhouettes remove the finger detail the model relies on.

Frequently Asked Questions