RC Surface v2.1.0

RC Surface

Q: Can I map from the phone?

Yes. MAP, pick a control, then Bind or Trigger Note.

Extension for Ableton Live 12 that serves a control surface to the browser of a phone or tablet: pads, knobs, faders, XY pads, mixer, transport and track recording. Each control can move a Live parameter, play a MIDI note or launch a Session clip. The phone motion sensors, microphone and camera can be mapped the same way.

Live 12.4.5+ Suite (Beta) · Safari on iOS 15.4+ or Chromium on Android, held in landscape · same local network · nothing is installed on the phone.

PERF: Performance page: pads, XY pads and LFO/Square modulators.
MIX: Mixer page: eight knobs and eight faders.
SNP: Snapshot page: stores the state of the surface and recalls it.
SNS · AUD · VID: Sensor pages: phone motion, microphone analysis and camera hand tracking.
REC: Recording page: arms tracks, records takes and sets a range between two markers.
MAP: Assigns a control or a sensor to one of three targets: Bind (a Live parameter), Trigger Note (a MIDI note) or Launch Clip (a Session clip).

Download 2.1.0 Source & licence

New in 2.1.0

REC: three views

The REC page has three views: Record, Arrangement and Clip.

CFG mode

Tap CFG in the header, then tap a pad, knob, fader or XY pad to edit its settings.

Network tab

The Network tab shows the address the QR code uses; USE pins one and FIND SETUP lists each with its own QR code.

Local certificate

A certificate authority of this computer signs the server certificate for localhost and private network addresses.

AUT

Tap AUT to record and loop XY pad gestures.

Recording and playback have not yet been verified in Live on macOS. Full list of changes: release notes.

Signal chain

┌─ NODE 0 ─────────────────────────────────┐
│ PHONE BROWSER          TOUCH IMU MIC CAM │◀─┐
│ PERF · MIX · SNP · SNS · AUD · VID · MAP │  │
└────────────────────┬─────────────────────┘  │
       HOP 1  ▼  HTTPS · SELF-SIGNED · WSS    │
┌─ NODE 1 ─────────────────────────────────┐  │
│ RC SURFACE SERVER                 v2.1.0 │  │
│ SMOOTH · CURVE EVAL · MORPH · SNAPSHOT   │  │
└────────────────────┬─────────────────────┘  │
       HOP 2  ▼  EXTENSIONS SDK · LOCAL       │
┌─ NODE 2 ─────────────────────────────────┐  │
│ EXTENSIONS SDK          LOCAL HOST API   │  │
│ PARAMETERS · MIXER · TARGET CATALOG      │  │
└────────────────────┬─────────────────────┘  │
       HOP 3  ▼  LIVE OBJECT MODEL · IN PROC  │
┌─ NODE 3 ─────────────────────────────────┐  │
│ ABLETON LIVE 12.4.5                SUITE ├──┘
│ SESSION · DEVICES · MIXER · TRANSPORT    │
└──────────────────────────────────────────┘
▼ CONTROL, OUTBOUND      ◀ STATE, INBOUND

Clock source

Mode	Needs	What it provides
AbletonOSC	AbletonOSC	Beat-accurate: metronome, subdivisions, swing, phase.
SDK	nothing	BPM simulator off Live's tempo. No beat phase.
Free	nothing	Internal clock. Not locked to Live's tempo.

AbletonOSC is needed only for Deep Sync, and installs separately as a Control Surface with UDP 11000/11001 free. Without it everything else works. Long-press SYNC for clock source, subdivisions, phase, swing and shapes.

Surface map

PERF MIX SNP SNS AUD VID REC MAP TRN AUT CFG

The plan figures are schematics drawn for the 2.0.0 layout, not screenshots.

Fig. 1 PERF: the layout as built Scale none Proj. plan

N	Region	Qty	What it is
1	Page tabs	7	PERF, MIX, SNP, SNS, AUD, VID, REC. MAP overlays the current page; AUT selects XY loops in PERF; CFG configures controls.
2	Pad grid	12	Four across, three down. Behaviour comes from the pad mode, not the pad.
3	Pad mode	4	A momentary, B hold, C toggle, D burst. Sets all twelve; CFG overrides one pad.
4	XY pads	2	Two independent `XY 1` / `XY 2` panels: Static or Physics, with friction, bounce, speed and trails. Each axis maps separately.
5	Toggles T1–T8	8	Waveform (Sine, Triangle, Ramp Up, Ramp Down, Square) set in CFG. Horizontal drag sets rate, vertical sets depth. Square includes swing and ratchet. Hz in FREE, beat divisions in SYNC.
6	PERF UTIL	6	CAP arms capture; CAP+1–4 saves; 1–4 recalls; OFF stops morph, pads, LFOs and Squares and centers XY, without touching MIX, sensors, audio, vision or transport.
7	AUT · XY gesture loops	2	AUT selects XY1/XY2; their separate AUT buttons open settings without recording. SYNC captures 1–5 bars from Live’s fresh playing clock; FREE captures 1–30 seconds in steps of one second. Play/Stop are explicit; manual touch overrides a playing loop until release. Saved points never auto-play after reconnect. This loops control data, not native Arrangement automation.
8	Live panel	8	BPM, transport (PREV/PLAY/NEXT), TRN, MAP, AUT, CFG. AUT is a header mode on PERF.
9	Sync block	3	SYNC toggles SYNC/FREE (hold for Deep Sync Settings, Clock source), ⚙ settings, STAGE locks surface to current page.

Fig. 2 MIX: eight knobs and eight faders Scale none Proj. plan

N	Region	Qty	What it is
1	Page tabs	7	PERF, MIX, SNP, SNS, AUD, VID, REC. MAP overlays the current page; AUT selects XY loops in PERF; CFG configures controls.
2	Knobs	8	`knob-1..8`, four across by two down. Relative vertical drag.
3	Faders	8	`fader-1..8`, one row. The thumb moves exactly as far as the finger.
8	Live panel	7	BPM, transport (PREV/PLAY/NEXT), TRN, MAP, CFG. No AUT on MIX.
9	Sync block	3	SYNC toggles SYNC/FREE (hold for Deep Sync Settings, Clock source), ⚙ settings, STAGE locks surface to current page.

Fig. 3 SNP: eight stored states and vector morph Scale none Proj. plan

N	Region	Qty	What it is
1	Page tabs	7	PERF, MIX, SNP, SNS, AUD, VID, REC. MAP overlays the current page; AUT selects XY loops in PERF; CFG configures controls.
2	Morph slots	8	Whole controller states in a four-by-two grid; recall or morph to one slot.
3	Performance controls	3	`CAPTURE`, `CLEAR SLOTS`, transition 0.1–5.0 s or synced, and Grid/Vector.
4	Vector morph pad	1	Blends slots 1 TL, 2 TR, 3 BL and 4 BR at once, weighted by the point.
8	Live panel	6	BPM, transport (PREV/PLAY/NEXT), TRN, MAP. No AUT or CFG on SNP.
9	Sync block	3	SYNC toggles SYNC/FREE (hold for Deep Sync Settings, Clock source), ⚙ settings, STAGE locks surface to current page.

Fig. 4 SNS: motion and orientation axes Scale none Proj. plan

N	Region	Qty	What it is
1	Page tabs	7	PERF, MIX, SNP, SNS, AUD, VID, REC. MAP overlays the current page; AUT selects XY loops in PERF; CFG configures controls.
2	Motion	6 ax	`GX GY GZ` angular rate, `AX AY AZ` acceleration, signed around a centre line.
3	Orientation	3 ax	Yaw, pitch and roll as numeric angles.
4	Local view	2	Shows or hides the two banks on this phone only; the sensors keep running.
5	Axis key	3 ax	Shows which phone axis each channel name refers to.
8	Live panel	6	BPM, transport (PREV/PLAY/NEXT), TRN, MAP. No AUT or CFG.
9	Sync block	4	SYNC toggles SYNC/FREE (hold for Deep Sync Settings, Clock source), ⚙ settings, CALIBRATE sets neutral reference (posture on SNS, level on AUD, lighting on VID), STAGE locks surface.

Fig. 5 AUD: audio input and twelve descriptors Scale none Proj. plan

N	Region	Qty	What it is
1	Page tabs	7	PERF, MIX, SNP, SNS, AUD, VID, REC. MAP overlays the current page; AUT selects XY loops in PERF; CFG configures controls.
2	Audio input	1	Pick the browser default or a permitted input, then enable capture. It never starts by itself.
3	Input level	1	Input RMS, normalized 0..1. Not perceived loudness.
4	Signal history	1	Live 2.5-second history: Amplitude, Attacks, Tone, Texture, Bands or All.
5	Detector knobs	10	SENS, RELEASE, CURVE, group GAIN and SMOOTH. Note divisions in SYNC, milliseconds in FREE.
6	Analysis window	1	WINDOW x1/x2/x4 trades spectral detail for response time.
7	Built-in audio detectors	12	Twelve 0..1 mapping sources: sensor.audio.transient, sensor.audio.kick, sensor.audio.snare, sensor.audio.brightness, sensor.audio.centroid, sensor.audio.flux, sensor.audio.flatness, sensor.audio.spread, sensor.audio.rolloff, sensor.audio.low, sensor.audio.mid, sensor.audio.high.
8	Live panel	6	BPM, transport (PREV/PLAY/NEXT), TRN, MAP. No AUT or CFG.
9	Sync block	4	SYNC toggles SYNC/FREE (hold for Deep Sync Settings, Clock source), ⚙ settings, CALIBRATE sets neutral reference (posture on SNS, level on AUD, lighting on VID), STAGE locks surface.

Fig. 6 VID: camera, learned poses, detectors Scale none Proj. plan

N	Region	Qty	What it is
1	Page tabs	7	PERF, MIX, SNP, SNS, AUD, VID, REC. MAP overlays the current page; AUT selects XY loops in PERF; CFG configures controls.
2	Camera	1	On/off, confidence, recognition mode, and the live preview.
3	Learned poses	3–15	Starts with G1–G3; + ADD POSE grows to fifteen cards. Capture 3–6 examples per pose; X removes a card, CLR keeps it. RETAKE/KEEP review isolated or conflicting captures without changing recognition.
4	Built-in detectors	4	Open, Fist, Pinch and Victory, opt-in and untrained. Rotation belongs to Victory.
5	Direct MAP / CLUTCH	2	MAP X/Y/Z, PALM and FACE are continuous mapping inputs. CLUTCH X/Y/Z is relative and freezes when the pinch opens.
6	Palm mapping inputs	2	PALM size and signed FACE are shown raw; MAP uses normalized 0–1 values. FACE is palm/back orientation, not facial tracking. Missing/invalid samples follow Safe loss.
8	Live panel	6	BPM, transport (PREV/PLAY/NEXT), TRN, MAP. No AUT or CFG.
9	Sync block	4	SYNC toggles SYNC/FREE (hold for Deep Sync Settings, Clock source), ⚙ settings, CALIBRATE sets neutral reference (posture on SNS, level on AUD, lighting on VID), STAGE locks surface.

Fig. 7REC: Arrangement / CLIPScale noneProj. plan

This drawing is the 2.0.0 schematic: two views, round level dials and the STAGE button. The current REC page has three views (Record, Arrangement and Clip) on one shared capture bar, with a volume fader on each track row. The table below describes each part; the User Guide describes it in full.

ARRANGEMENT CLIP

N	Region	Qty	What it is
1	RECORD / ARRANGEMENT / CLIP switch · CLICK	3	One fixed mode bar switches the three local views, with Record as the default. The capture bar (REC / STOP / CLICK) is shared by all three and keeps the same place in every view. CLICK is shared with TRN: during setup, pre-roll, REC and playback it switches Live’s metronome, following the metronome state Live last reported.
2	REC / STOP / CLICK · BACK TO ARR	4	STOP has priority during preparation and pre-roll. REC turns Punch In OFF for a direct start and ON for a pre-roll, and restores Punch In only when it still holds the value REC wrote. BACK TO ARR appears when Session overrode Arrangement, and returns explicitly once every Session clip is idle.
3	PRE-ROLL and PASSES / Until STOP	2	OFF starts at the marker. 1/2/4 play earlier bars with click before native Punch In opens at the marker. This browser preference does not set Live’s read-only native count-in.
4	Markers / bar.beat / Zoom	1	Markers are prepared in Live and appear in the catalog; the phone creates none. Choosing a marker is a local selection that never seeks Live’s transport. Zoom and scroll preserve musical scale; both end cards have gutters.
5	START / END markers	2	Choose a START marker; the END is the Next marker, an explicit marker, or Until STOP for an open-ended capture. Live confirms the region before REC, and the range stays frozen while the capture is running.
6	Track rows / volume fader / ARM MULTI	4	ARM MULTI keeps inputs cumulative. EXCLUSIVE disarms others only when you press a track ARM. REC verifies exactly the visible armed set; STOP keeps armed inputs intact. A group row keeps its own fader, and arming it arms the armable descendants inside it.
7	CLIP / Session scene	1–16 bars	CLIP replaces the timeline with a paged Session grid on the same shared track rows. Choose target scene; LENGTH (1/2/4/8/16 bars) and QUANTIZE (Global/None/1 bar/1 beat) set in the panel. REC shows queued, recording or playing; STOP stops only the tracks REC started, keeps ARM and the global transport, and stays pending until Live confirms the stop. BACK TO ARR explicitly returns once every Session clip is idle, without Pause/Play. No existing-clip overdub or take/comp browser.
8	Live panel	5	BPM, transport (PREV/PLAY/NEXT), TRN only. Entering REC closes MAP; no AUT or CFG.
9	Sync block	3	SYNC toggles SYNC/FREE (hold for Deep Sync Settings, Clock source), ⚙ settings, STAGE locks surface to current page.

Fig. 8 MAP: armed strip, action mode, then target picker Scale none Proj. plan

N	Region	Qty	What it is
1	Underlying page	1	MAP is an overlay, not a tab. The page underneath stays visible until a control is picked.
2	Armed strip	1	Shows only `MAP ARMED / TAP A CONTROL / DONE`, and goes once a control is tapped.
3	Editing overlay	1	A 400 px right-side pane owns the mapping editor.
4	Action modes	3	`BIND` drives a Live parameter with curve and range; `TRIGGER NOTE` plays a MIDI note through `RC-Midi-Receiver.amxd`; `LAUNCH CLIP` fires Session clips through AbletonOSC.
5	Target filter	1	Searches target names; `Selected in Live` narrows to the current track and device.
6	Hierarchy	3 levels	Track → Device → Parameter, plus Song/Main/Master and Return groups.
7	Live selection shortcut	1	Uses Live's selected track and device as a filter; it picks no parameter for you.
8	Live panel	7	BPM, transport, TRN, active MAP overlay, CFG over current page.
9	Sync block	3	SYNC toggles SYNC/FREE (hold for Deep Sync Settings, Clock source), ⚙ settings, STAGE locks surface to current page.

Fig. 9 TRN: Transport Lite overlay Scale none Proj. plan

N	Region	Qty	What it is
1	Main transport row	4	PLAY, STOP, PREV, NEXT with large targets. Mirrors Live's native playback state.
2	CLICK, Quantization, REFRESH	3	CLICK shared with REC and never locked; the Global Quantization is read from the Live Set when the overlay opens and written back to it; REFRESH resyncs the transport state.
3	Searchable locators	1	List of Arrangement cue points with search filter. Tapping a locator jumps Live’s transport to it, and playback keeps running.
4	Status and sync badge	1	Badge shows SYNCED, SDK or FREE timing. Notice line appears only when AbletonOSC is missing, a change is unconfirmed, or in viewer mode.
8	Live panel	8	BPM, transport, active TRN overlay, MAP, AUT, CFG.
9	Sync block	3	SYNC toggles SYNC/FREE (hold for Deep Sync Settings, Clock source), ⚙ settings, STAGE locks surface to current page.

Fig. 10 AUT: XY gesture loop settings and capture Scale none Proj. plan

N	Region	Qty	What it is
1	Header mode activation	1	AUT is a contextual header mode available only on PERF. Tapping AUT arms gesture capture on either XY pad.
2	Gesture capture & loop	2	Tapping an XY initiates a 3–2–1 red pulse countdown; red border indicates active recording; captured movement loops immediately upon completion.
3	Hold for settings	1	Holding an XY pad for 600 ms without moving (or right-clicking on desktop) opens its settings overlay without recording.
4	Duration and timing	2	SYNC offers 1–5 bars (default 2 bars); FREE offers 1–30 seconds in 1 s steps (default 10 s). Changing length or timing mode discards that XY's recording.
5	Playback, takeover & safety	4	Record/replace, Play, Stop, and Clear (with confirmation). Manual touch overrides loop until release; white target square indicates takeover point. Does not write Live automation.
8	Live panel	8	BPM, transport, TRN, MAP, active AUT mode, CFG.
9	Sync block	3	SYNC toggles SYNC/FREE (hold for Deep Sync Settings, Clock source), ⚙ settings, STAGE locks surface to current page.

Fig. 11 CFG: per-control configuration overlay Scale none Proj. plan

N	Region	Qty	What it is
1	Configuration mode (CFG)	1	Per-control settings: pad mode, eight toggle slots, Static/Physics on either XY. PERF and MIX only.
2	Pad mode overrides	12	Overrides global A–D mode for a single pad: A, B and C set drag travel (Very short to Long) and response (Linear/Fine/Fast); A touch level; B tap without drag keeps or zeroes; C on level; D timing (40–10000 ms or musical beat, attack, peak).
3	Toggle & LFO parameters	8	Select waveform (Sine, Triangle, Ramp Up, Ramp Down, Square), timing mode and subdivision. Advanced settings expose phase and swing/ratchet for Square.
4	XY Static & Physics	2	Toggle between Static and Physics per XY pad. Physics settings: friction (0.002–0.05), bounce (0–0.95), speed (0.1–4×). Reset removes override.
5	Hold to edit points	2	MAP response curve points and Burst envelope points edit only after holding for ~0.6 s: indicator ring appears, drag to move, release to set. Quick taps and scrolls never shift points. Double tap resets.
8	Live panel	8	BPM, transport, TRN, MAP, AUT, active CFG mode.
9	Sync block	3	SYNC toggles SYNC/FREE (hold for Deep Sync Settings, Clock source), ⚙ settings, STAGE locks surface to current page.

What each family does, from touch to Live, is in Controls.

.surface-tabs{display:none} .surface-deck{display:block} .surface-sheet{visibility:visible;opacity:1;pointer-events:auto} .surface-sheet + .surface-sheet{margin-top:30px}

Controls

01Performance 02Continuous controls 03Snapshots 04Motion & audio 05Vision 06Mapping & MIDI

Pads, LFOs, squares

A returns to zero, B keeps what you leave, C latches, D fires one burst, set per pad in CFG. Each of the eight LFO slots can use Square; choose its own clock in CFG or follow the global SYNC/FREE setting.

VALUE ↑TIME →TOUCH = ▼ · UP = △

A · RELEASE

touchup

drag sets levelUP → immediate zero

B · HOLD

SAVEDKEEP

drag edits levelUP → hold new value

C · TOGGLE

TAP ONTAP OFF

two-touch latchhold + drag edits ON

D · BURST

70 ms attack450 ms release

one-shot A–Rdrag peak .15–1 · length follows the grid in SYNC

BEAT

1234

SYNC / FREErepeat at beat division

XY, knobs, faders

Each XY panel independently selects Static or Physics, with its own friction, bounce and speed. Knobs and faders track the finger from the current value and never jump to the touch point.

1 · RELATIVE VERTICAL DRAGKNOB · FADER

The touch point is never the value. The control starts from its current value

value

same
length

finger

At touch-down dy is 0, so nothing moves. The knob crosses its range in 150 px; a fader travels as far as the finger.

value = start + dy / rangePx KNOB 150 · MACRO KNOB 220 · FADER 150

2 · PHYSICSXY 2

Release does not stop it

travelledwhere it keeps goingpad edge · rebounds at ×0.75

CONFIGURABLE SPEED v ×= 0.988 each frame · stops under 0.01

Snapshots and vector morph

Capture stores every mapped control into one of eight slots. Recall interpolates to it over a set time. Vector XY blends slots 1–4 by finger position.

STORE A STATE · RECALL RETURNS TO IT OVER THE SET TIME · BLEND BETWEEN FOUR STATES

CAPTURE / RECALL

LIVE STATE

VOL

CUT

SEND

PAN

CAP

EIGHT SLOTS

12345678

CURRENTSLOT 3

RECALL = timed interpolation · Free 0.1–5.0 s or Sync in beats

VECTOR MORPH

BLENDED OUTPUT

A .12

B .54

C .08

D .26

four corner states → weighted blend

Motion, orientation, audio

CALIBRATE makes the current posture the reference. AUD turns sound into control: level, envelope, gate, attacks and twelve descriptors, graphed over the last 2.5 seconds.

MOTION SETS A REFERENCE · AUDIO MAPS SOUND TO CONTROL VALUES

MOTION

CALIBRATE

AX AY AZ · GX GY GZ

AUDIO INPUTLIVE

RMS0.42

ENV0.38

SIGNAL / 2.5sLIVE

RMSENV

SENS0.65

band risedrag ↑↓

RELEASE45 ms

pulse tail10–500

WINDOW · 1024 @48kx1x2x4

SYNC follows Live BPM from 1/128 to 1/1, triplets and dotted included; FREE keeps its own milliseconds. Low, mid and high are K-weighted loudness (ITU-R BS.1770), in LU.

Built-in audio detectors

ATTACKS

TRANSIENT0.82

KICK0.64

SNARE0.10

TONE

BRIGHTNESS0.36

CENTROID3000 Hz

ROLLOFF 95%5000 Hz

TEXTURE

FLUX0.25

FLATNESS0.08

SPREAD3000 Hz

BANDS

LOW0.24

MID0.18

HIGH0.09

cyan = instantaneous RMS · green = smoothed envelope

Vision: hand, poses, clutch

Hand X/Y/Z, PALM size and FACE palm orientation map continuously. A held pinch engages the clutch; release freezes its axes. G1–G15 and the four built-in detectors provide outputs. Missing/invalid samples follow Safe loss.

PINCH CLUTCH: YOU CAN RELEASE THE PINCH AND RESUME FROM THE SAVED VALUE

1 · ENGAGE

the two tips touchcontact rises past .75the clutch takes hold

2 · MOVE

the hand travels in spaceX, Y and Z follow itfrom where you started

3 · RELEASE

the fingers partcontact drops below .55the value stays put

PINCH AGAIN: THE CLUTCH RESUMES FROM THE SAVED VALUE

CONTACT = gap between landmark 4 and landmark 8, divided by palm size ARM at .75 · RELEASE below .55 · latched in between

sensor.vision.pinch_x · _y · _z relative, never absolute

TEACH A POSE: STEP SEQUENCE

1 · ADD POSE

+ ADD POSE creates custom gesture slots up to G15 (starts with G1–G3). Each pose is specific to the active hand.

2 · CAP EXAMPLES

Press CAP 3–6 times with hand still, slightly varying angle and finger curvature. TEST unlocks at 3 takes.

3 · SILENT TEST

TEST performs silent live matching, showing "nearest take N · X% of radius" without firing live MIDI or parameter outputs.

4 · EDIT & REVIEW

DEL LAST deletes the last take; CLR resets all takes to retrain; X removes the card. RETAKE/KEEP warnings are advisory and do not alter recognition.

5 · BUILT-IN DETECTORS & PRESETS

Built-in detectors Open, Fist, Pinch, and Victory are opt-in and require no training; Victory includes wrist rotation. CONF sets Low/Medium/High threshold; presets include Precision, Balanced (default), Performance, and Flexible.

6 · SAFE LOSS & MAPPING

Tracking loss settings (VID → ⚙): Hold indefinitely, Hold with timeout (0.1–30 s, default 10 s), or Release on loss. Map poses in MAP as sensor.vision.gesture.N (momentary 0/1 gate, re-arms on hand release).

Built-in detectors: Open / Fist / Pinch / Victory Pinch Clutch engages relative control on sensor.vision.pinch_x · _y · _z

MAP, MIDI, and mapping surfaces

Phone MAP and panel Mappings give each control one of three actions: bind a Live parameter, trigger a MIDI note, or launch a Session clip, with curve, range, takeover and idle settings.

ONE SIGNAL PATH · SOURCE DOMAIN → 0..1 SHAPE → TARGET DOMAIN

SOURCE

sensor.audio.centroid

range you set

MEASURED SENSOR DOMAIN

TRANSFORMINPUT → OUTPUT

NORMALIZE

inMin – inMax → 0..1

CLAMP0..1

SOURCE RANGE

RESPONSE CURVE

LINEAR · EXPONENTIAL · LOGARITHMIC · S-CURVE

TARGET RANGE

Song Tempo

20–300 BPM

184 BPM

SCALED LIVE PARAMETER

POLICYTAKEOVERSCALEPICKUPHOLDSAFE LOSSRELEASESMOOTHING

ACTION BINDTRIGGER NOTELAUNCH CLIP LIVE PARAMETER / MIDI / SESSION CLIP

SIX-STEP MAPPING SEQUENCE: FROM TOUCH TO LIVE

1 · PICK TARGET

Open MAP, tap any control, then choose target via Track → Device → Parameter picker (+ Song/Main/Master & Returns; search filter; "Selected in Live").

2 · NORMALIZE SOURCE

Set input range In min / In max to map to 0..1. CLAMP bounds values outside the range to prevent jumps.

3 · RESPONSE CURVE

Choose Linear, Exponential, Logarithmic, or S-Curve shape, with Drive, Comp, and Smooth. Hold points for ~0.6 s to edit; double tap resets.

4 · TARGET RANGE

Set target Out min and Out max in native Live units. Scaling modes: Auto, Linear, or Geometric for frequency/pitch controls.

5 · POLICIES

Takeover / pickup policies prevent jumps on reconnection; Safe loss defines Hold vs Release behavior on signal drop; idle timeout restores baseline.

6 · BIND · TRIGGER NOTE · LAUNCH CLIP

BIND writes Live parameters via the Extensions SDK. TRIGGER NOTE sends MIDI notes through RC-Midi-Receiver.amxd manually placed on the MIDI track (pitch/octave/velocity, timing Now/Next Beat/Bar, gate Short/While Held; keyboard MIDI passes through directly). LAUNCH CLIP fires Session clips through AbletonOSC: launch one clip, advance a list or stop the track, at the quantization Live reports.

.control-tabs{display:none} .control-layout,.control-deck{display:block} .control-sheet{position:static;visibility:visible;opacity:1;pointer-events:auto} .control-sheet + .control-sheet{margin-top:20px}

Situations

Scenario A · two performers

XY pad -> return effects

A colleague opens the phone and moves the return effects while the soloist keeps playing.

xy-1.x→distortion Dry/Wetxy-1.y→reverb send

Scenario B · knob and fader

Knob -> Auto Filter cutoff, fader -> reverb return

A knob opens the Auto Filter and a fader moves the reverb return.

knob-1→Auto Filter cutofffader-1→reverb return

Scenario C · pinch clutch

Pinch -> Auto Filter cutoff

Standing, with no table: pinch the palm towards the camera, move the hand and open the fingers.

sensor.vision.pinch_z→Auto Filter cutoff

Scenario D · instrument motion

Rotation rate -> Echo feedback

A phone strapped to a guitar turns the swell into a rotation-rate gesture.

sensor.motion.gx→Echo feedback

Scenario E · acoustic input

Audio envelope -> Saturator Drive

A phone near the microphone: the envelope follows the loudness of the phrases.

sensor.audio.envelope→Saturator Drive

Installation

Clip 1 Install and first pairing, narrated mp4

Install the extension

Double-click the .ablx from the 2.1.0 release to request installation in Live.

Open the panel

In Live, open Extensions → RC Surface → Show panel. It shows the QR code, the connection status and the real URL if a port is taken; the link drawn in Signal chain.

Connect the phone

Scan the QR code, install the certificate authority of this computer once from the plain page at the HTTPS port + 1, and hold the phone in landscape.

Arm the sensors

Enable Motion, Audio or Vision as needed. Raw camera and microphone never leave the phone. Only numeric values do.

Map a control to Live

Tap MAP, pick a highlighted control, then bind it to a Live parameter or a trigger note. Notes need the bundled RC-Midi-Receiver.amxd on the target MIDI track: Live cannot insert Max for Live devices through the SDK.

Optional: Deep Sync

Install AbletonOSC as a Control Surface with UDP 11000 and 11001 free, then long-press SYNC to pick the clock; see Clock source.

Troubleshooting

Phone cannot find the host

Same Wi-Fi. Live allowed through the firewall. Rescan the current QR, or open the Network tab in the panel and press FIND SETUP.

AbletonOSC not detected

Install it separately as a Live Control Surface.

Vision will not start

Accept the certificate, grant camera, reload.

Audio is silent

Grant microphone, then tap the audio control again.

No Extensions menu in Live

Check the Live 12.4.5+ Suite Extensions host is enabled, then restart Live.

Frequently asked

Which Live versions?

12.4.5+ Suite (Beta).

Android and iOS?

iOS 15.4+ Safari, or Chromium on Android.

Can I map from the phone?

Yes. MAP, pick a control, then Bind or Trigger Note.

What about latency?

Controls are coalesced at 30 Hz; audio has its own bounded path. Measure end to end on your network.

Camera and microphone?

Processed inside the phone's browser. Never uploaded.

Reference documents

Installation Guide

Read INSTALL.md

User Manual

Read USER-GUIDE.md

Customization & Dev

Read CUSTOMIZATION.md

FAQ

Read FAQ.md

Security Threat Model

Read SECURITY.md

Read PRIVACY.md

░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░