hc_gesture — Hand Gestures to MIDI
✨ New in homecrate 1.12. If you don’t see it yet, update from the App Store.
hc_gesture turns your hands into a MIDI controller. It’s an AUv3 music effect you load on any track in Homecrate, and it passes audio through completely untouched — its output isn’t sound, it’s control data. The front camera watches your hands, a gesture you make (a fist, a pinch, a thumbs-up) becomes a MIDI CC message or a note, and that message is routed straight into the plugin chain on the track hc_gesture is sitting on, or out to external MIDI gear. It exists for the moment when both hands are already busy: you’re holding a guitar, you’re at the keys, you’re on the mic, and you still want to open a filter, punch a delay in, or swell a pad without reaching for the screen.
🟢 Never done camera-based control before? Go to Quick Start — one assignment, one ARM tap, and you’re performing in about a minute. The single idea to hold on to is that a gesture is a shape you hold, and what you do with it while you hold it is set by the mode.
🔵 Coming from a MIDI controller rig? Think of it as an 11-button, 2-hand controller where every button also has an endless encoder built in. Each row in the assignment table is one mapping — gesture + hand + CC/note + mode. ROTATE mode is the encoder: it reads your wrist angle relative to the angle you started the gesture at, so every grab begins at mid-range. Output goes to the host track’s plugin slots on the same path as hardware MIDI input, plus optional external MIDI out.
Quick Start
- Load homecrate: hc_gesture on the track that holds the plugin you want to control (see Loading & Removing Plugins) — it’s in the Effects tab
- Tap the loaded slot to open the editor. The top half is the live camera preview; the bottom half is the control panel
- Under ASSIGNMENTS, tap + ADD — a new row appears and opens
- Set Gesture to Pinch (Index), tap the RIGHT hand chip, leave CC, and tap the ROTATE mode chip
- Drag CC # to the controller number your target plugin listens on (74 is the conventional filter-cutoff CC)
- Tap ARM. The state chip counts down through STABILIZING — get your hands into position while it runs
- When the chip reads LIVE, pinch your right thumb and index finger together and twist your wrist. The value climbs and falls on the overlay chip, and your plugin follows
Tap DISARM when you’re done. It’s panic-safe: everything held is released as it disarms.
Loading hc_gesture in Homecrate
hc_gesture registers as an AUv3 music effect and appears in the Plugin Browser’s Effects tab as homecrate: hc_gesture. See Loading & Removing Plugins for the general steps and AUv3 Plugins Overview for how slots work.
Because it passes audio through bit-for-bit, slot position doesn’t matter for tone. What matters is which track it’s on: the MIDI it generates is delivered to every plugin slot on that track, exactly like MIDI arriving from a hardware controller. Put hc_gesture and the synth you want to control on the same track and the routing is already done — there is nothing to configure.
You can load several instances on different tracks. Each one keeps its own independent assignment table, so a gesture rig for your synth track and a different rig for your guitar track can coexist. Arming, however, is global — see ARM / DISARM.
Note: hc_gesture opens a video-only capture session. It never adds an audio input, so your recorder monitoring, armed tracks, amp sims and audio session are completely undisturbed by having it running. It needs camera permission the first time you open the editor or arm it.
The Interface
[SCREENSHOT: hc_gesture editor — camera preview with tracking overlay on top, control panel below]
The editor is split in half. The camera preview occupies the top; the control panel scrolls underneath it in this order:
| # | Panel | What it’s for |
|---|---|---|
| 1 | Header | Plugin name, state dot, and the current state: DISARMED / STABILIZING _n_s / LIVE |
| 2 | Status strip | The gesture(s) you’re currently holding, and the last MIDI message sent |
| 3 | ARM row | The ARM / DISARM button and a count of how many assignments you have |
| 4 | STABILIZE | The countdown length, 1–15 seconds |
| 5 | EXTERNAL MIDI | An ON / OFF toggle for sending to external MIDI gear |
| 6 | ASSIGNMENTS | The mapping table — one row per gesture → MIDI mapping, plus + ADD |
The Camera Preview & Tracking Overlay
[SCREENSHOT: Camera preview — fingertip reticles, hand boxes, label chip with live value]
The preview shows the front camera, mirrored so you see yourself the way you’d see yourself in a mirror. Drawn over it is the tracking overlay, which is both the confirmation that tracking is working and the tool you use to dial mappings in.
What’s on it:
| Element | Meaning |
|---|---|
| Fingertip reticles | A small square with a centre dot on each tracked fingertip — up to five per hand |
| Hand box | A stroke-only frame around each tracked hand. Faint while merely tracking; when a gesture locks it gains a fluttering ghost frame and a scan sweep |
| Gesture dot | A pulsing dot at the gesture point — the pinch midpoint for pinches, the box centre otherwise. Gains a crosshair and ring while a gesture is locked |
| Combined pinch frame | When a pinch locks, the two fingertip squares merge into one breathing double frame with a link line between thumb and finger |
| Connector line | A thin line joining the two hands when both are tracked |
| Label chip | A tag hanging above the gesture dot, with the gesture name and — on a tab below it — the live MIDI value being produced |
| Countdown ring | A sweeping arc in the centre of the frame while STABILIZING, emptying as the countdown runs out |
The overlay’s colour tells you the state at a glance:
| Colour | Meaning |
|---|---|
| White | Tracking your hands. Nothing is being sent (disarmed) |
| Amber | Stabilizing — the countdown is running, gestures register but nothing is sent |
| Mint green | LIVE — gestures are sending MIDI |
| Hot cyan | A pinch is locked. Cyan always wins over the state colour, so a lock is unmistakable |
Two hands are tracked at once. The camera session runs whenever the editor is open or while armed, and stops once you disarm and close the editor.
Header & Status Strip
[SCREENSHOT: Header row — state dot, hc·gesture, STABILIZING 4s]
The header carries a coloured dot and the state text:
| State | Text | Dot |
|---|---|---|
| Disarmed | DISARMED | Dim white |
| Stabilizing | STABILIZING 4s (counts down) | Amber |
| Live | LIVE | Mint |
Below it the status strip shows the gesture (or gestures, separated by ·) you’re currently holding on the left — no gesture when your hands are empty — and the last message sent on the right, in the form CC74=96, NOTE 60 ON, or NOTE 60 OFF. Both are useful sanity checks while building a rig: if the left side is blank when you’re clearly holding a shape, hc_gesture isn’t recognising it — change your lighting or exaggerate the shape; if the left side is populated but the right side is stale, you’re not LIVE yet.
ARM / DISARM
[SCREENSHOT: ARM row — ARM chip and the "3 ASSIGNED" counter]
hc_gesture doesn’t listen until you tell it to. Tap ARM to start the sequence; the button becomes DISARM (amber) as soon as it’s armed. To the right, a counter shows how many assignments the current instance holds — 0 ASSIGNED through however many rows you’ve built.
Two behaviours worth knowing:
- Arming is global. There is one arm state for the whole app, not one per plugin. Arming from any instance’s editor puts every loaded hc_gesture instance live at once, and disarming stops them all. Each instance still only sends its own assignments, to its own track.
- DISARM is panic-safe. Every held note gets its note-off and every open gate gets its off value at the moment you disarm, so nothing can be left stuck on. The same release happens automatically if the camera stops.
STABILIZE
[SCREENSHOT: STABILIZE slider set to 5 s]
STABILIZE (1–15 s, default 5 s) is the length of the countdown between tapping ARM and going LIVE.
It exists for a practical reason. In the seconds after you tap ARM, your hands are doing all sorts of things — leaving the screen, picking up an instrument, settling onto the keys. Every one of those shapes is a gesture as far as hc_gesture is concerned, and without a countdown they’d all fire. During STABILIZING, tracking and recognition carry on as normal (the overlay tracks, the countdown ring sweeps, ROTATE values move) but nothing is sent. When the countdown ends the chip turns to LIVE.
There’s a second piece of protection at the go-live instant: any gesture you’re already holding when the countdown ends is suppressed until you release it. If you happen to be gripping a pick in a way that reads as a fist when LIVE hits, that fist won’t fire — you have to let go and make it again.
If you keep triggering things while you get set up, raise STABILIZE. 8–10 seconds is not excessive if you need to sit down and shoulder a guitar.
Note: With several hc_gesture instances loaded, arming uses the longest STABILIZE value among them, since there’s a single global countdown.
EXTERNAL MIDI
[SCREENSHOT: EXTERNAL MIDI row with the ON chip lit]
Off by default. Turn it ON and every message this instance generates is also sent out to your external MIDI gear, in addition to going to the host track — a hardware synth, a desktop DAW, a lighting rig, anything connected to your device over MIDI.
Note: EXTERNAL MIDI is an on/off toggle only. There is no destination picker — when it’s on, messages go to every connected MIDI destination. If you need to target one device, filter by MIDI channel at the receiving end.
ASSIGNMENTS
[SCREENSHOT: ASSIGNMENTS list — collapsed rows with gesture name, hand badge and summary line]
Each row is one mapping: this gesture, on this hand, sends this MIDI, in this way. Tap + ADD to create one (it opens straight into the editor); tap a row’s summary line to expand or collapse it; DELETE at the bottom of an expanded row removes it.
A collapsed row shows the gesture name, a mint hand badge when the hand isn’t ANY, and a summary line reading e.g. CC 74 · ROTATE or NOTE C4 · GATE, with · INV appended when INVERT is on.
Many assignments can share one gesture. A single right-hand fist can fire a note, stamp a CC and open a gate at the same time — just add three rows with the same gesture and hand.
Every field
| Field | Values | Default | Appears when |
|---|---|---|---|
| Gesture | Any of the 11 (menu) | Fist | Always |
| Hand | ANY / LEFT / RIGHT | ANY | Always |
| Kind | CC / NOTE | CC | Always |
| Mode | START / END / GATE / ROTATE | START | CC only |
| CC # | 0–127 | 1 | Kind = CC |
| NOTE | 0–127, displayed as a note name (C-1 … G9) | 60 (C4) | Kind = NOTE |
| VEL | 1–127 | 100 | Kind = NOTE |
| CHAN | 1–16 | 1 | Always |
| MIN | 0–127 | 0 | Kind = CC |
| MAX | 0–127 | 127 | Kind = CC |
| INVERT | on / off | off | Kind = CC |
| SHAPE | LIN / EXP / LOG / S-CRV / SWELL | LIN | CC + ROTATE |
| SMOOTH | 0–100 % | 50 % | CC + ROTATE |
| RANGE | 30° – 360° | 150° | CC + ROTATE |
MIN and MAX are just the two ends of the range — you can set MIN above MAX and it works exactly the same, because the pair is sorted before use. INVERT mirrors the value inside that range, so an inverted 0–127 sweep runs 127→0, and an inverted trigger sends MIN where it would have sent MAX.
CC or NOTE
CC assignments send Control Change messages and get the full set of modes and ranges.
NOTE assignments are a key trigger and nothing else: the row switches to GATE behaviour automatically and the mode chips are replaced by the fixed label ON @ START · OFF @ END. Note-on with your chosen VEL the moment the gesture is recognised, note-off the moment you release it. MIN/MAX, INVERT, SHAPE, SMOOTH and RANGE don’t apply and aren’t shown.
The 11 Gestures
[SCREENSHOT: Gesture menu open on an assignment row]
| Gesture | The shape | Recognised as |
|---|---|---|
| Open Palm | Flat hand, everything out | Four fingers extended and thumb extended |
| Fist | Closed hand | No fingers extended, thumb not extended |
| Point | Index finger only | Index extended, nothing else |
| Two Fingers | Peace sign | Index + middle, ring and pinky down |
| Three Fingers | Index + middle + ring | Those three, pinky down |
| Four Fingers | All fingers, thumb tucked | Four fingers extended, thumb not extended |
| Thumbs Up | Thumb out, fist otherwise | Thumb extended, no fingers extended, thumb tip clearly above the wrist |
| Pinch (Index) | Thumb tip touching index tip | Thumb and index tip together or very nearly |
| Pinch (Middle) | Thumb tip touching middle tip | ” |
| Pinch (Ring) | Thumb tip touching ring tip | ” |
| Pinch (Pinky) | Thumb tip touching pinky tip | ” |
Pinches win. A pinch always takes priority over the other shapes, so you never have to worry about one being misread as a fist while your other fingers are curled. If two pinches could qualify at once, the tighter one is the one that fires.
Note the distinction between Open Palm and Four Fingers — the only difference is whether your thumb is out. If you find yourself getting one when you wanted the other, that’s the thing to change.
Hand selection
Every assignment is pinned to a hand:
- ANY — either hand fires it
- LEFT — only your left hand
- RIGHT — only your right hand
Left and right are as you experience them. The front camera mirrors you, and Homecrate corrects for that, so LEFT means the hand you think of as your left. You never have to reason about the mirror.
Because gesture and hand are independent, 11 gestures give you far more than 11 controls — a left-hand fist and a right-hand fist are completely separate mappings.
ANY and two hands: an ANY assignment is latched by whichever hand triggers it first. While that hand holds the gesture, the other hand making the same shape is ignored for that row, so you can’t double-fire a gate or stack two note-ons.
The Four Trigger Modes
Mode is what separates a switch from an expression pedal. It’s available on CC assignments; NOTE assignments are always start/end.
START
Fires once, the instant the gesture is recognised, sending MAX (or MIN with INVERT on). Nothing more happens while you hold it, and nothing happens when you let go.
Use it to stamp a value: launch something, jump a parameter to a setting, hit a one-shot.
END
Fires once, when you release the gesture — the same value START would have sent, just at the other edge. Pair one START row and one END row on the same gesture and you get a “set on grab, reset on release” pair with two different CCs.
END won’t fire from a gesture that was already being held when the countdown ended — it needs a clean start while LIVE, so a shape you happened to be holding through arming can never fire a spurious release.
GATE
MAX while you hold the gesture, MIN when you release. A momentary footswitch made out of your hand: hold a fist, the delay send is up; open your hand, it’s back down. With INVERT on, the two swap, so the gesture ducks instead of boosts.
This is the mode to reach for by default for anything binary — bypass, mute, a send in or out.
ROTATE
See ROTATE in Depth — it deserves its own section.
ROTATE in Depth
ROTATE is a knob you grab out of the air. Make the gesture, then twist your hand as if you were turning a physical control, and hc_gesture streams continuous CC as you turn. It’s the reason the plugin exists.
[SCREENSHOT: Assignment row in ROTATE mode showing SHAPE, SMOOTH and RANGE]
It always starts in the middle
The detail that makes this playable rather than a novelty: rotation is measured relative to the angle your hand was at the moment the gesture locked, not to some absolute “up”. Whatever your wrist happens to be doing when you pinch, that orientation becomes the centre of the range.
So the value at the instant of the grab is always the middle of MIN…MAX. Twist one way to go up, the other way to come down, release to leave it wherever you stopped. You never have to pre-position your hand, and you can grab the same control twice in a take from two completely different arm positions and get identical behaviour.
The consequence to design around: a fresh grab starts at mid-range, so if you want to sweep a filter from closed to open, grab, twist down to the bottom first, then sweep up — or narrow MIN/MAX so mid-range is already musically useful.
Twist your wrist, not your fingers
Rotation reads the orientation of your whole hand, so the motion to make is the one you’d make turning a physical knob: keep the shape and rotate the wrist. Waggling your fingers inside the shape does nothing.
That includes pinches — it’s tempting to think a pinch rotates by rolling the two fingertips against each other, but with your thumb and fingertip touching there’s nothing there to read. Pinch, then turn the whole hand.
RANGE (30° – 360°, default 150°)
How far you have to twist for the full MIN→MAX travel. Because you start at the centre, RANGE is the total span — a 150° RANGE means roughly ±75° either side of where you grabbed.
| RANGE | Feel |
|---|---|
| 30°–90° | A flick. Fast, dramatic, hard to place precisely |
| 120°–180° | The sweet spot for most controls. 150° is the default for good reason |
| 240°–360° | A long, deliberate sweep with lots of resolution — good for slow filter opens, but you’ll run out of wrist |
If a mapping pins at 127 before your wrist is comfortable, raise RANGE. If you can’t reach the top, lower it.
SMOOTH (0 – 100 %, default 50 %)
How much the motion is averaged before it’s sent. At 0 % the response is immediate and slightly jittery; at 100 % it’s creamy and noticeably laggy. 50 % suits nearly everything. Raise it for slow atmospheric sweeps; lower it for anything that needs to feel like a switch you can snap.
SHAPE
The response curve applied between MIN and MAX — see Envelope Shapes.
A twist survives a stumble
A camera loses sight of a hand every so often — especially mid-rotation, when your own fingers pass in front of each other. hc_gesture is built to ride through that rather than break the take:
- A brief loss doesn’t drop your gesture. The value keeps going and nothing is released.
- If a gesture does drop and you make it again straight away, the twist resumes where you left off rather than re-centring. You get one continuous move instead of the classic “big value, sudden snap to the middle, big value again” glitch.
Wait longer than a moment before re-making it and it counts as a fresh grab, starting from the middle again — which is usually what you’d want by then anyway.
Rate and feedback
Values are sent only when they actually change, and the stream is rate-limited, so even a fast twist can’t flood the plugin chain.
The value is always computed, even while disarmed or stabilizing — only the sending is gated. That means you can open the editor, make the gesture and watch the number on the overlay chip move before you ever arm. It’s the fastest way to tune RANGE and SMOOTH.
Envelope Shapes
SHAPE decides how your twist maps to the value between MIN and MAX. Five curves:
| Shape | Curve | Feel |
|---|---|---|
| LIN | Straight line | Predictable and neutral. Start here |
| EXP | Slow at first, then rushes | Anything that should “open up” late — filter cutoffs, sends, drive |
| LOG | Fast at first, then eases in | Anything that should react instantly then settle — volume, wet mix |
| S-CRV | Gentle at both ends, quick through the middle | The most “designed” feel; soft landings at both extremes |
| SWELL | Rises to the top mid-travel and falls back | A shape rather than a ramp — twist through it for an automatic swell and return |
These are deliberately the same five curves used by the timeline’s automation shape presets — identical math, identical names (see Automation Lanes & Shape Presets). A swell you draw on an automation lane and a swell you perform with your hand behave the same way, so performed moves and drawn moves sit together consistently in a mix.
INVERT mirrors the result inside MIN…MAX, so INVERT + EXP gives you a fast-then-easing descent.
Worked Example 1 — Pinch and twist a filter cutoff
The classic: grab a filter out of the air with your right hand and open it.
- Set up the track. Load your instrument (homecrate synth, say) on a track, and add hc_gesture to the same track. That’s the whole routing job.
- Find the CC number. Check which CC the target listens on for cutoff. CC 74 is the conventional one for brightness.
- Add the assignment. Tap + ADD and set:
| Field | Value |
|---|---|
| Gesture | Pinch (Index) |
| Hand | RIGHT |
| Kind | CC |
| Mode | ROTATE |
| CC # | 74 |
| CHAN | Match the target plugin’s channel (1 unless you’ve changed it) |
| MIN / MAX | 0 / 127 |
| SHAPE | EXP |
| SMOOTH | 50 % |
| RANGE | 180° |
- Rehearse before arming. With the editor open and still DISARMED, pinch and twist. The overlay label chip goes hot cyan and the value tab under it counts. Adjust RANGE until a comfortable wrist rotation covers the whole span.
- Arm. Tap ARM and use the amber countdown to pick up your instrument.
- Perform. At LIVE, pinch, twist clockwise to open, anticlockwise to close, release to hold the value where you left it.
- Refine. If it feels twitchy, raise SMOOTH. If only part of the sweep is musically useful, narrow MIN/MAX to that part rather than fighting the curve — e.g. MIN 40 / MAX 110.
[SCREENSHOT: Pinch locked — hot cyan double frame with the live CC value chip mid-range]
Worked Example 2 — A left-hand GATE for a delay send
Now the switch half of the rig, on the hand you aren’t sweeping with.
- On the same track, tap + ADD again for a second row.
- Set it up as:
| Field | Value |
|---|---|
| Gesture | Fist |
| Hand | LEFT |
| Kind | CC |
| Mode | GATE |
| CC # | The delay’s mix/send CC on your plugin |
| CHAN | Same as row one |
| MIN / MAX | 0 / 100 |
| INVERT | off |
- Hold a left fist and the send jumps to 100; open your hand and it drops to 0. Turn INVERT on and the same fist ducks the send to 0 instead — handy for a hand-operated mute.
- For a hand-played note instead of a CC, switch Kind to NOTE: the mode chips are replaced by
ON @ START · OFF @ END, and the fist plays your chosen NOTE at your chosen VEL for as long as you hold it — see MIDI Tracks Overview and The Piano Roll Editor for what receives it.
Left hand punches the delay in and out, right hand sweeps the filter, and neither hand ever touched the screen.
What Gets Saved
The whole configuration — every assignment row, the STABILIZE length and the EXTERNAL MIDI toggle — is stored in the plugin’s state and saved with the project, alongside everything else about that track. Reopen the project and your rig is exactly as you left it; the only thing you need to do again is tap ARM.
Each instance keeps its own table and its own identity, so removing one hc_gesture from one track never disturbs another.
Reliability & Limits
hc_gesture is built for a real room, not a demo, and a few behaviours are worth knowing so you can trust it:
- Distance doesn’t matter much. A gesture reads the same close to the camera as it does across the room — you don’t have to find a “sweet spot” and stay in it, and you don’t have to re-tune a rig because you stood up.
- It doesn’t chatter. A held gesture stays held. Sitting right on the edge of a pinch won’t make it flicker on and off, and a momentary wobble in the tracking can’t fire a trigger or drop one you’re holding.
- Which hand is which stays stable. A fast move won’t swap your left and right mid-gesture, so a rotation you started with one hand can’t jump onto the other.
- The overlay glides rather than blinks. Boxes and reticles ease toward your hands and hold their last position briefly through a stumble, so a steady overlay is a good sign even if a frame was missed.
- A hand it can’t see clearly is ignored, not guessed at. If part of your hand is out of frame or obscured, hc_gesture would rather report nothing than report the wrong shape.
None of that makes camera tracking infallible. Backlighting, very dim rooms, gloves, busy backgrounds, and hands moving very fast across the frame all cost accuracy. Test your rig before you rely on it in a take.
Tips
- Assign the hand you’re not using. If you fret with your left, put everything on RIGHT so a chord shape can never fire a trigger.
- Pinches are the most reliable gesture for ROTATE. They’re an unambiguous shape, they’re tested first, and the hot-cyan lock gives you visual confirmation before you start twisting.
- Tune ROTATE while disarmed. Values are computed the whole time and shown on the overlay; you only need to be LIVE when you want them to actually go somewhere.
- Watch the value chip, not the parameter. It’s far easier to see what’s being sent than to infer it from what you’re hearing.
- Start at RANGE 150° and move in steps of 30°. Too small and the parameter is impossible to place; too large and you run out of wrist.
- Narrow MIN/MAX instead of fighting the curve. If only 40–110 sounds good, map 40–110 and keep LIN.
- Use GATE for anything binary and save ROTATE for the one or two things that genuinely need to sweep. Two continuous mappings at once is about the limit of what’s performable.
- Raise STABILIZE to 8–10 s if you need to sit down and get an instrument into position.
- Stack rows on one gesture when you want a single move to do several things — a fist that fires a note and opens a gate is two rows, not a compromise.
- DISARM before you talk with your hands. It’s instant and panic-safe.
- Light your hands, not your background. Hand tracking wants your hands to be the clearest thing in frame.
Next steps
- General plugin loading: Loading & Removing Plugins
- How plugin slots and chains work: AUv3 Plugins Overview
- Where the MIDI can go: MIDI Tracks Overview
- The same five curves, drawn instead of performed: Automation Lanes & Shape Presets
- Editing MIDI clips: The Piano Roll Editor