Time-setting of sound-objects

Bernard Bel

Time-setting of sound-objects: a constraint-satisfaction approach. Invited paper, Workshop on Sonic Representations and Transforms. INTERNATIONAL SCHOOL FOR ADVANCED STUDIES (ISAS), Trieste, 26-30 October 1992.

Abstract

This paper deals with the sched­ul­ing of “sound-objects”, here­by mean­ing pre­de­fined sequences of ele­men­tary tasks in a sound proces­sor, with each task mapped to a time-point. Given a struc­ture of sound-objects com­plete­ly ordered in a phase dia­gram, an “instance” of the struc­ture may be obtained by com­put­ing the dates at which each task should be exe­cut­ed. Time-setting the struc­ture amounts to solv­ing a sys­tem of con­straints depend­ing on (1) met­ric and topo­log­i­cal prop­er­ties of sound-objects, (2) con­texts in which they are found, and (3) para­me­ters relat­ed to the per­for­mance itself (“smooth” or “stri­at­ed” time, speed, etc.). This may require relocating/truncating objects or delay­ing part of the sound-object struc­ture. A constraint-satisfaction algo­rithm is intro­duced, the time com­plex­i­ty of which is O(n.k) in most cas­es, where n is the num­ber of sequences and k the max­i­mum length of a sequence. In the worst case it remains bet­ter than O(n².k³). Other fields of appli­ca­tions are pro­posed, includ­ing mul­ti­me­dia per­for­mance and computer-aided video editing.

Excerpts of an AI review of this paper (Academia, June 2025)

Summary of the Work

The man­u­script details a com­pu­ta­tion­al approach to the sched­ul­ing and instan­ti­a­tion of “sound-objects,” which are defined as sequences of ele­men­tary tasks in a sound proces­sor. The crux of the work is an algo­rithm that solves a sys­tem of con­straints aris­ing from topo­log­i­cal and met­ri­cal prop­er­ties of these sound-objects, as well as from con­tex­tu­al per­for­mance para­me­ters. The algo­rithm bal­ances local changes (relo­ca­tion and trun­ca­tion of objects) against glob­al shifts (delays) to ensure that all con­straints with­in sequences of sound-objects are met. In addi­tion to musi­cal appli­ca­tions, such an approach could be extend­ed to oth­er time-based media, includ­ing mul­ti­me­dia sce­nar­ios and video editing.

Core Contributions

Explicit Discussion of Time and Constraints

• The paper frames time not just as a uni­ver­sal met­ric but as a struc­ture of dis­crete “time streaks” or beats. In this sense, the work bridges for­mal rep­re­sen­ta­tions of time (from dis­crete puls­es to “smooth” con­tin­u­ous flow) with performance-related constraints.
• The for­mu­la­tion of con­straints (e.g., over­lap­ping or dis­joint objects) is sys­tem­at­i­cal­ly addressed through defin­able prop­er­ties (e.g., overBeg, overEnd, truncBeg, truncEnd), which are then used in a constraint-satisfaction process.

Algorithmic Design and Complexity Analysis

• The step-by-step “Locate()” func­tion is well described with a clear flow­chart, detail­ing how the algo­rithm attempts cor­rec­tions through local or glob­al drifts, as well as par­tial trun­ca­tions of the sound-objects.
• A worst-case time com­plex­i­ty of O(n²·i³) (under cer­tain con­di­tions) is estab­lished, yet it often per­forms more effi­cient­ly, show­ing that this method is viable for real-world musi­cal and mul­ti­me­dia applications.

Applicability to Broader Contexts

While the exam­ples focus pri­mar­i­ly on musi­cal objects, the paper rec­og­nizes poten­tial appli­ca­tions in any set­ting where time-based data need to be sched­uled, such as multiple-screen video edit­ing or live mul­ti­me­dia per­for­mances. This gen­er­al­iz­able frame­work under­scores the ver­sa­til­i­ty of the approach.

Strengths

• Comprehensive Motivations: The work demon­strates a sol­id under­stand­ing of music com­po­si­tion needs and how strict time-based approach­es can ben­e­fit from greater flexibility.
• Thorough Explanations: The back­ground and ratio­nale for each com­po­nent of the constraint-satisfaction process are well artic­u­lat­ed, guid­ing read­ers step-by-step.
• Foundational for Further Development: The text pro­vides ample con­cep­tu­al and algo­rith­mic foun­da­tions for more advanced tim­ing algo­rithms, poten­tial­ly inspir­ing oth­er constraint-based sched­ul­ing research in music and mul­ti­me­dia domains.

Overall, the paper pro­vides a struc­tured and flex­i­ble method for rep­re­sent­ing and manip­u­lat­ing time-synchronized objects, show­ing promise not only for music com­po­si­tion but also for a wide vari­ety of mul­ti­me­dia appli­ca­tions requir­ing coor­di­nat­ed play­back and sequenc­ing. The detail in con­straint cod­i­fi­ca­tion, strate­gies for local and glob­al cor­rec­tions, and empha­sis on com­pu­ta­tion­al com­plex­i­ty will be par­tic­u­lar­ly valu­able to those work­ing with large-scale or com­plex per­for­mance setups.

Download this paper