A phone browser, turned into a controller surface for Ableton Live. The surface is served from your own machine over local HTTPS; the phone connects to it over the same network. Nothing is installed on the phone.
Held in landscape. Seven pages, twelve pads, six knobs, six faders, two XY pads, three sensor inputs. This sheet is the operator's reference for build 0.6.0.
| Version | 0.6.0 |
|---|---|
| Requires | Ableton Live 12.4.5 or later, Suite edition |
| Phone | iOS 15.4+ Safari, or a modern Chromium browser on Android |
| Held | Landscape. Portrait is refused by the surface. |
| Network | Local HTTPS. LAN only. No internet needed. |
| Licence | PolyForm Noncommercial 1.0.0, source-available |
| Source | ntworm.github.io/ableton-rc-surface |
| Notice | Independent project. Not affiliated with Ableton AG. |
| Host | Ableton Live 12.4.5 Suite |
| Bridge | RC Surface 0.6.0, running inside Live |
| Served | HTTPS, on a port the bridge picks automatically |
| Socket | WSS, same port |
| Certificate | Self-signed, generated on first launch, current LAN IPs in the SAN list |
| OSC out | UDP 11000 |
| OSC in | UDP 11001 |
| Sync | AbletonOSC, deep — optional, see 2.1 |
| Reach | One local network. No outbound connections. |
Control inventory is in 4.0. This section covers the connection only.
┌─ NODE 0 ─────────────────────────────────┐ │ PHONE BROWSER TOUCH IMU MIC CAM │◀─┐ │ PERF · MIX · SNP · SNS · AUD · VID · MAP │ │ └────────────────────┬─────────────────────┘ │ HOP 1 ▼ HTTPS · LOCAL CA · WSS │ ┌─ NODE 1 ─────────────────────────────────┐ │ │ RC SURFACE SERVER v0.6.0 │ │ │ SMOOTH · CURVE EVAL · MORPH · SNAPSHOT │ │ └────────────────────┬─────────────────────┘ │ HOP 2 ▼ OSC / UDP · 127.0.0.1:11000 │ ┌─ NODE 2 ─────────────────────────────────┐ │ │ ABLETONOSC REMOTE SCRIPT │ │ │ UDP IN :11000 · UDP OUT :11001 │ │ └────────────────────┬─────────────────────┘ │ HOP 3 ▼ LIVE OBJECT MODEL · IN PROC │ ┌─ NODE 3 ─────────────────────────────────┐ │ │ ABLETON LIVE 12.4.5 SUITE ├──┘ │ SESSION · DEVICES · MIXER · TRANSPORT │ └──────────────────────────────────────────┘ ▼ CONTROL, OUTBOUND ◀ STATE, INBOUND
Both directions run at 30 Hz and both stay on the LAN. Outbound, a touch becomes a socket frame, the server resolves it against the curve and the current snapshot, and it leaves as OSC. Inbound carries tempo, beat and transport state, plus the current values of mapped parameters, so a fader moved in Live moves on the phone. Latency depends on router, distance, interference and host load, so no single figure applies. Measure on the target network before relying on it in performance.
| Mode | Needs | What you get |
|---|---|---|
| AbletonOSC | AbletonOSC | Beat-accurate. Transport metronome, subdivisions, swing, phase. |
| SDK | nothing | BPM simulator off Live's tempo. No beat phase. |
| Free | nothing | Internal clock. LFOs and stutters are not locked to Live’s tempo. |
AbletonOSC is needed only for Deep Sync. Without it the surface still works — pads, knobs, faders, sensors, mapping and snapshots are unaffected. It installs separately, is added in Live as a Control Surface, and requires UDP 11000 out and 11001 in to be free.
Long-press SYNC on the phone for Deep Sync Settings: clock source, subdivisions, phase offsets, swing, and the shape used by LFOs and stutters. The header button itself only toggles SYNC and FREE; the three-state SYNCED / SDK / FREE readout lives inside the transport overlay behind TRN.
| N | Region | Qty | What it is |
|---|---|---|---|
| 1 | Page tabs | 6 | PERF, MIX, SNP, SNS, AUD, VID. MAP is the seventh page and opens over the current one. |
| 2 | Pad grid | 12 | Four across, three down. Behaviour comes from the pad mode, not from the pad. |
| 3 | Pad mode | 4 | A momentary, B hold, C toggle, D burst. Applies to all twelve pads at once. Carries no heading on screen. |
| 4 | XY pads | 2 | Captioned XY 1 and XY 2 (Physics), each with a live 0.50 / 0.50 readout. Each axis maps on its own. |
| 5 | LFOs | 4 | L1 to L4. Each toggle carries its own value bar. |
| 6 | Stutters | 4 | S1 to S4. Rate and shape come from Deep Sync settings, not from the button. |
| 7 | PERF UTIL | 6 | CAP captures, slots 1 to 4 recall, OFF exits. Six cells, two across. |
| 8 | Live panel | 6 | BPM readout plus five buttons: previous locator, play and pause, next locator, TRN, MAP. Header, on every page. |
| 9 | Sync block | 4 | SYNC, its settings gear, CALIBRATE, and STAGE. Header, on every page. |
MODE A PERF UTIL ┌──────┬──────┬──────┐ │ 1 ▓▓▓│ 2 ░ │ 3 ▓▓ │ ├──────┼──────┼──────┤ │ 4 ░ │ 5 ▓▓▓▓│ 6 ░ │ ├──────┼──────┼──────┤ │ 7 ▓▓ │ 8 ░ │ 9 ▓▓▓│ ├──────┼──────┼──────┤ │10 ▓ │11 ░ │12 ▓▓ │ └──────┴──────┴──────┘ A MOMENTARY B HOLD C TOGGLE D BURST
Twelve pads. Four across, three down on the device; stacked here to fit the column. Bar levels are illustrative.
MIX TAB 6 CH F1 F2 F3 F4 ┌─┐ ┌─┐ ┌─┐ ┌─┐ │░│ │░│ │▓│ │░│ │░│ │░│ │▓│ │░│ │▓│ │░│ │▓│ │░│ │▓│ │▓│ │▓│ │▓│ └─┘ └─┘ └─┘ └─┘ KNOBS K1-K6 SHARE THE SAME BANK AND THE SAME MAP TABLE.
Six knobs, six faders. Both banks are on the MIX tab.
XY 2 (PHYSICS) ┌────────────────┐ │ │ │ · │ │ · · │ │ · │ │ · ▓ │ │ │ └───┴────┴───────┘ 0.50 / 0.50 X ▶ AXIS 1 MAPPING Y ▶ AXIS 2 MAPPING · TRAIL ▓ HEAD
Two XY pads. Each axis maps on its own, so one gesture can drive two unrelated parameters. The second runs the physics model.
| N | Module | Qty | Page | Sends | Target in Live |
|---|---|---|---|---|---|
| 00 | Performance pads | 12 | PERF | MIDI | Clip, scene, or a trigger note |
| 01 | Pad modes A–D | 4 | PERF | local | Pad behaviour only |
| 02 | XY pads | 2 | PERF | OSC | Two parameters per pad |
| 03 | LFOs L1–L4 | 4 | PERF | OSC | Any mapped parameter |
| 04 | Stutters S1–S4 | 4 | PERF | OSC | Any mapped parameter |
| 05 | PERF UTIL column | 6 | PERF | local | CAP, slots 1–4, OFF |
| 06 | Knobs | 6 | MIX | OSC | Device parameter |
| 07 | Faders | 6 | MIX | OSC | Track volume |
| 08 | Morph slots | 8 | SNP | OSC | A whole controller state |
| 09 | Transition time | 1 | SNP | local | 0.1 s to 5.0 s, default 1.0 s |
| 0A | Vector morph pad | 1 | SNP | OSC | Blends slots 1–4 by corner: TL TR BL BR |
| 0B | Motion | 6 ax | SNS | OSC | GX GY GZ, AX AY AZ |
| 0C | Orientation | 3 ax | SNS | OSC | Yaw, pitch, roll |
| 0D | Audio input | 1 | AUD | OSC | Pitch, note, BPM, envelope, gate, clarity, bend |
| 0E | Single-hand vision | 1 | VID | OSC | Open, fist, pinch, victory, fingers |
| 0F | Map picker | 1 | MAP | local | Assigns any row above |
| 10 | Header live panel | 6 | header | OSC | BPM, prev locator, play and pause, next locator, TRN, MAP |
| 11 | Header sync block | 4 | header | OSC | SYNC, settings, CALIBRATE, STAGE |
Sends is the transport a row uses: OSC reaches Live through AbletonOSC, MIDI is a note the host receives, local never leaves the phone. Every OSC row is assigned in MAP mode — row 0F. Rows 10 and 11 sit in the header and are reachable from every page, which is why their Page column reads header rather than a tab.
Live's tempo, mirrored. Read-only.
Previous locator, play and pause, next locator. Controls Live's transport from the phone.
Full-screen overlay: PLAY, STOP, PREV, NEXT, REFRESH, a locator list and a search. Carries the SYNCED / SDK / FREE readout. Flashes green on beat 1, blue on the others.
Opens the mapping picker as an overlay on the current page. It has no tab of its own.
A two-state toggle: the button itself reads SYNC or FREE. Long-press for Deep Sync Settings. See 2.1.
Opens Deep Sync Settings directly, without the long-press.
Re-zeroes motion and orientation to the phone’s current position.
Locks the surface to the current page, preventing accidental page changes during performance.
Link state and the phone's name. Tap to rename.
CAP to capture, direct snapshot slots 1–4 to recall, and OFF to exit. These four recall directly, without the transition applied to the SNP morph slots.RC-Midi-Receiver.amxd is reused if it is already on the track, and falls back to manual insert on failure. Pads, LFOs, stutters, XY axes, knobs and faders all support trigger-note mappings.SYNC for clock source, subdivisions, phase offsets, swing, and the shapes used by LFOs and stutters. See 2.1.| Tab | Page | What is on it | Rows |
|---|---|---|---|
| PERF | Performance | Pad grid, pad mode A–D, two XY pads, LFOS L1–L4, STUT S1–S4, PERF UTIL | 00–05 |
| MIX | Mixer | Six knobs, six faders | 06–07 |
| SNP | Snapshots | Eight morph slots, capture and clear, transition time, morph mode, vector morph pad | 08–0A |
| SNS | Sensors | Motion GX GY GZ and AX AY AZ; orientation yaw, pitch, roll; local view | 0B–0C |
| AUD | Audio | Pitch, note, BPM, envelope, gate, clarity, bend | 0D |
| VID | Vision | Camera, confidence, learned poses G1–G3, built-in detectors, direct map | 0E |
| MAP | Mapping | Hierarchical picker. Overlays the current page; no tab of its own. | 0F |
A controller surface for Live 12.4.5+ Suite, source-available under PolyForm Noncommercial 1.0.0. Intended for a local, auditable Live Extension workflow with mapping controlled from the phone.
| Feature | Ableton RC Surface | Other controller options |
|---|---|---|
| Install on phone | Browser page served by the extension | Usually a native app or web app |
| Cost | Source-available for noncommercial use | Varies by app |
| Protocol | Live's Extensions SDK | Often OSC, MIDI, or a bridge app |
| Mapping workflow | Binds parameters and trigger notes from the phone | Depends on each app |
| DAW support | Live 12.4.5+ Suite only | Often broader DAW support |
| Phone sensors | Motion, mic analysis, optional camera tracking | Depends on the app and platform |
| Updates | Manual .ablx releases | Depends on each release process |
| Source code | Available, PolyForm Noncommercial 1.0.0 | Varies by project |
Steps 01 to 03 establish the link shown in 2.0. Nothing responds until the phone is connected.
Download Ableton-RC-Surface-0.6.0.ablx. Double-click it and Ableton Live prompts to install it under User Library / Extensions. Restart Live if it was already running.
If the Extensions menu does not appear, check that the Live 12.4.5+ Suite Extensions host is enabled, then restart Live.
Inside Live, open Extensions → Ableton RC Surface → Show panel. The panel shows a Performance QR code and the local connection status. The bridge picks an available port automatically, so the URL is not fixed between runs.
Scan the Performance QR code. Accept the self-signed HTTPS certificate warning once. The QR encodes the current URL. Hold the phone in landscape; portrait shows an orientation notice instead of the surface.
The twelve pads take their behaviour from the mode bar: A momentary, B hold, C toggle, D burst. Move between PERF, MIX, SNP, SNS, AUD and VID to reach performance controls, knobs and faders, snapshots, sensors, audio and vision.
Enable Motion and orientation, Audio microphone analysis, or Vision single-hand tracking as needed. The browser asks for permission locally. Raw camera and microphone streams are never sent anywhere — only numeric control values leave the phone.
Tap MAP on the phone, select a highlighted control, then bind it to a Live parameter or to a MIDI trigger note. The picker groups Song, Main and Master, normal tracks, return tracks, devices and parameters, so the target context stays visible. Trigger notes use the bundled RC-Midi-Receiver.amxd.
Install AbletonOSC separately and add it in Live as a Control Surface, keeping UDP 11000 and 11001 free. Long-press SYNC on the phone to pick the clock source. Without it the surface runs on the SDK BPM simulator or its own internal clock — see 2.1.
Same Wi-Fi. Live allowed through the firewall. Rescan the current QR.
The port changes between runs, so an old URL will not work.
Install it separately as a Live Control Surface.
Keep UDP 11000 and 11001 free. Only Deep Sync needs it.
Accept the certificate, grant camera, reload.
All three are required, in that order.
Grant microphone, then tap the audio control again.
Browsers require a user gesture before capture starts.
Check the Live 12.4.5+ Suite Extensions host is enabled, then restart Live.
12.4.5+ Suite (Beta).
It is built on Live's native Extensions SDK. Live 11 and standard editions have no Extensions host.
iOS 15.4+ Safari, or Chromium on Android.
iOS 14.5 and below cannot get past local self-signed certificate warnings.
No. Everything runs on your local network.
The MediaPipe hand-tracking runtime ships with the extension and is served locally.
Yes. MAP, pick a control, then Bind or Trigger Note.
The picker groups Song, Main and Master, tracks, returns, devices and parameters.
30 Hz both ways over local Wi-Fi. No universal number.
Router, distance, interference and host load all affect it. Measure on the target network.
Processed inside the phone's browser. Never uploaded.
Only numeric control values reach Live, over the local WebSocket.
No. Independent and source-available.
Not affiliated with, sponsored by, or endorsed by Ableton AG.
Only for Deep Sync.
Without it the controller runs on the SDK BPM simulator or its own internal clock.
v0.6.0. tsc --noEmit clean; the production build produces a working .ablx.
Smoke-tested in Live 12.4.5+ Suite (Beta). Full validation on physical iOS and Android devices is outstanding.