Ableton RC Surface is a source-available Ableton Live 12.4.5+ Suite extension v0.5.8.4 that turns a phone browser into a low-latency performance, mix, mapping, and sensor controller. It ships with mobile MAP mode, MIDI trigger notes, a TRN transport overlay, AbletonOSC deep sync, single-hand camera hand tracking, and hierarchical target picking. The core controller runs on your local Wi-Fi; hand tracking loads MediaPipe from a CDN when used.
Not affiliated with Ableton AG. Actively maintained, audited line by line, free for noncommercial use under PolyForm Noncommercial 1.0.0.
Drop the .ablx into your User Library, restart Live, and pair your phone in seconds — no app install, no hub.
v0.5.8.4 - every card below ships in the current release.
Four pad behaviors (release, hold, toggle, burst) plus a PERF UTIL column with CAP, direct snapshot slots 1-4, and OFF for instant live-performance state changes.
A mixer-style page for common Live controls, plus a dedicated MIX tab inside the phone with six mappable knobs and six mappable faders.
Two-axis controls for filter moves, sends, or macros. X and Y axes are exposed as separate mappable channels on the phone editor.
Select highlighted controls on the phone and bind them. The target picker is grouped by Song / Main / Master, normal tracks, return tracks, devices, and parameters. Tap Selected in Live to query the host and pre-fill the search.
Motion, orientation, and microphone analysis are built in. Camera hand tracking is single-hand by design (open, fist, victory, pinch) and loads MediaPipe from a CDN.
Eight morphable snapshot slots for interpolating between controller states, plus four direct capture / recall slots on the PERF UTIL column. Mapped Live values follow the transition while the morph is running.
Map a phone control to a note on a MIDI track. The included RC-Midi-Receiver.amxd is reused if already on the track and falls back to manual insert on failure. Pads, LFOs, stutters, XY axes, knobs, and faders all support trigger-note mappings.
Tap TRN in the header to open a full-screen transport: Play / Stop, prev / next locator, locator jumps, and a dynamic locator search. The TRN button flashes green on beat 1 and blue on the others when sync is engaged.
Long-press SYNC to open Deep Sync Settings: Clock Source (AbletonOSC, SDK BPM Simulator, Free / Internal), subdivisions, phase offsets, swing, and shapes for LFOs and Stutters. Status badge shows SYNCED / SDK / FREE.
The mobile mapping editor includes a 2D response canvas with a moving dot for live input / output feedback while you tweak range and curve. Idle values and per-control rate are configurable.
A self-signed certificate is generated on first launch so the phone browser can request camera and microphone permissions. Current LAN IPs are included in the certificate SAN list.
A controller surface for Ableton Live 12.4.5+ Suite. Source-available under PolyForm Noncommercial 1.0.0, audited line by line, no telemetry, no account. It is not trying to be the broadest controller app - it focuses on tight Live Extensions SDK integration, mobile-first mapping, and transparency over feature breadth.
| Feature | Ableton RC Surface | Other controller options |
|---|---|---|
| Install on phone | Browser page served by the extension | Usually a native app or web app |
| Cost | Source-available for noncommercial use | Varies by app |
| Protocol | Uses Live's Extensions SDK | Often OSC, MIDI, or a bridge app |
| Mapping workflow | Can bind parameters and trigger notes from the phone | Depends on each app |
| DAW support | Built only for Live 12.4.5+ Suite | Often broader DAW support |
| Phone sensors | Motion, mic analysis, optional camera tracking | Depends on the app and platform |
| Updates | Manual .ablx releases | Depends on each app's release process |
| Source code | Source available under PolyForm Noncommercial 1.0.0 | Varies by project |
If you need a polished, proven controller today, an established app may be the better choice. RC Surface is for people who want to test a local Live Extension workflow and help shape the first public version.
A guided walkthrough from unzipped folder to a Live set that responds to your phone. Read once, then perform.
Open Ableton Extensions Manager, click the downloaded .ablx, then confirm. Move the unpacked folder anywhere under Documents/Ableton/User Library/ — the manager registers it for you. After install, quit Live entirely and reopen so the new extension is loaded.
Drag RC-Midi-Receiver.amxd from vendor/ onto any MIDI track in your Live Set. The device starts AbletonOSC (UDP 11000/11001) and a UDP MIDI listener (port 9000) automatically — no setup dialog. The device’s chain selector picks which chain receives the phone’s output.
Open the admin at http://<your-host>:4444/admin. It shows a QR code pointing at /phone. Scan it with your phone, or type the URL manually. The phone opens the PWA controller. Tap Connect. First connection takes ~1s (handshake + state snapshot).
The 8×8 pad grid has four modes, selected by the tabs at the top. A — Scene launcher (8×8 = 64 clips). B — Clip detail (stop, double, MIDI notes view). C — Macro map (8 macros per track, across 8 tracks). D — Custom user mapping. Modes are persisted per device via localStorage and survive reconnects.
The right rail exposes three sources: Motion (device orientation, 6DOF), Vision (camera + MediaPipe hand tracking, up to 60 FPS), and Audio (mic RMS + pitch + envelope via AudioWorklet). Toggle any source on — the phone asks for permission once, then streams values continuously.
In the admin, open the Mappings tab and pick a Live parameter (device knob, macro, track volume). Choose a sensor source on the phone side, then move the phone / camera / mic. The mapping records the range you sweep and saves it as a curve. Apply — the parameter now follows the sensor in real time.
RC-Midi-Receiver.amxd must be on a track in the current Set. Re-add it if you started a new live set.http://localhost:4444/phone on the phone itself, or enable HTTPS.Three steps once Live Extensions are enabled. The phone side opens in the browser, so there is no app-store install.
Download the Ableton-RC-Surface-x.y.z.ablx package. Double-click the file, and Ableton Live will automatically prompt you to install it under your User Library / Extensions folder.
Inside Live, **right-click on any Track Header or empty area in the Session View** to open the context menu, then select **Extensions → Ableton RC Surface → Show panel** (or press the keyboard shortcut Ctrl+Shift+A on Windows / Cmd+Shift+A on macOS). A pairing QR code will appear on screen.
Scan the QR code with your phone to open the controller page. Your phone browser may warn you about the self-signed HTTPS certificate. Accept the warning once, then test the connection before relying on it.
Setup notes, current limits, and privacy details for the v0.5.8.4 release.
cdn.jsdelivr.net unless they are already cached. On a fully offline studio network, camera tracking will be unavailable while the rest of the controller remains active.
SDK BPM Simulator or FREE / Internal clock modes.
npx tsc --noEmit is clean, the production build produces a working .ablx, and the package has been smoke-tested in Ableton Live 12.4.5+ Suite (Beta). Full validation on real iOS / Android devices is the next milestone.
Reference documents for installing, testing, troubleshooting, and understanding the current limitations.
Setup instructions for Windows, macOS, the local HTTPS warning, and first-run mapping checks.
Breakdown of pad modes, mobile MAP mode, LFO/stutter controls, mixer pages, sensors, and snapshot behavior.
Developer notes for mobile mapping, target routing, trigger notes, extending scripts, and adjusting the web interface.
Compatibility notes, diagnostics, and setup troubleshooting questions.
Notes on local network behavior, HTTPS setup, WebSocket endpoints, and current security assumptions.
Official statement documenting local media processing, MediaPipe CDN use for hand tracking, and zero telemetry practices.