Sketch MIDI from an audio phrase

Use sketch-midi-from-audio to make an isolated monophonic phrase playable with another instrument while keeping the audio original. For example: “Make a MIDI sketch of this bass loop on a new track, keep its timing and let me choose a synth.” This is a source-audio sketch with approximate pitches and boundaries, not exact transcription or a polyphonic audio-to-MIDI converter.

The workflow uses the all profile, an updated bridge and local ffmpeg/ffprobe. plan_audio_midi_sketch reads an exact warped Session or Arrangement audio clip. By default it analyzes the current loop when enabled, otherwise the playback markers. Supply startBeats and endBeats together for another region. These are absolute clip beats; an Arrangement region is not the global Arrangement time or the repeated/truncated occurrence heard on the timeline.

Regions can span at most 256 quarter-note beats and contain 0.256–30 seconds of source-file audio. Native beat-to-source conversion chooses the analysis window, and native source-to-beat conversion places each detected event. If the requested region projects beyond the file, sourceWindow.clampedToFile reports the clipped analysis window while the MIDI length retains the requested leading or trailing silence. Unwarped clips remain unchanged and require a separately scoped approach.

The result includes notes, noteEvidence, lengthBeats, source context, source-file metadata and detector boundary uncertainty. Note starts are relative to the requested region: its first beat becomes MIDI beat zero. Silence and unquantized native beat positions are preserved in the proposal. Empty detection stays empty; no placeholder notes are invented. Velocity is a chosen uniform value, defaulting to 100, rather than measured performance dynamics.

Analysis uses one selected source channel, defaulting to channel zero. It does not mix stereo channels or render clip pitch, re-pitch playback, groove, envelopes or devices. Pitches are nearest equal-tempered source notes at A4=440 Hz. Inspect clipPitch before matching a transposed clip, and resolve any octave, fine-tuning or rendered-output differences explicitly. Fast repetitions, vibrato, noisy attacks and polyphonic material can split, hide or misidentify notes. Listen and review the candidates before treating them as a musical part.

Choose an existing empty MIDI Session slot or authorize a new track. Resolve the instrument separately; MIDI notes alone do not imply audible output. Refresh the proposal after the destination is ready, preserve manual edits and compare its source context, metadata and notes before creating the sketch with create_midi_clip. The creation confirmation guards the destination and requested notes, not the audio source as one atomic transaction. Metadata checks are not file-hash verification, and a later source edit can still invalidate an earlier proposal.

Read back the complete MIDI notes and timing after creation. Match the source meter with set_clip_timing when needed, using its observed numerator and denominator. Keep the audio original and the chosen MIDI alternative. Audition, cleanup and saving each follow the request's scope; ask what the user hears before claiming that the result sounds right.