Making Interactive Sound

What are we going to do?

This course will introduce three platforms or development environments: Max, Processing and Arduino. We will then get each of them to talk to each other so that we can build interactive sound installations with visuals and physical controllers.

Throughout the sessions you will be able to design interactions between physical the world and audio and video generated by a computer. In each session you will have time to design and develop an idea of your own along with completing exercises to help inspire. By the end you will have produced a prototype of an installation that you will be able to further develop beyond the course using the tools you will have learned in the sessions.

Course Overview

• Session 1: Max
• Session 2: Processing
• Session 3: Arduino
• Session 4: Putting It All Together

Resources

Useful links to help you along.

Max

• Max 6
• Cycling 74 Community Site

Processing

• Processing

Arduino

• Getting started with Arduino
• Tom Igoe's Physical Computing Course
• Arduino Video Tutorials by Jeremy Blum

Session 1: Max

In this session you will generate audio/music in real-time and learn about different synthesis and audio processing techniques.

Introducing Max

• What is Max?

Programming in Max

• Messages and Objects
• Program Flow

MIDI and Messages in Max

• Generating MIDI Notes
• Metro and Toggle
• Sliders and Dials
• Random Numbers, Counters and Drunk
• Storing and Recalling Information
• Making Tidy Patchers and Encapsulation
• Scheduling Events
• Exercise 1
  • Create a Max patcher for making music that both:
    • schedules some notes automatically
    • allows the user to generate notes using buttons, sliders and dials

Max and Audio

• Audio Rate vs. Control Rate
• Generating Simple Waveforms
• Visualising Sound
• Volume and Tremolo
• Fades and ADSR Curves
• Filters
• Frequency Modulation and FM Synthesis
• Delay
• Audio File Playback
• Buffers and Playback
• Exercise 2
  • Create a piece of music using a Max patch that:
    • Can generate arbitrary notes using FM synthesis
    • Can playback pre-recorded audio
    • Applies some filters and effects (e.g. tremolo) to the output

Session 2: Processing

In this session you will learn how to generate and manipulate video and images in real-time.

Introducing Processing

• What is Processing
• Exercise 1
  • Open up Processing and go to Examples... under the Files menu
  • Run different examples to see what you can do.

Basic Drawing

• Getting Started with Processing
• Exercise 2
  • Draw a landscape using shapes and lines.
• Displaying Images with Processing
• Text
• 3D Graphics

Interactions

• Mouse Events in Processing
• Keyboard Events in Processing
• Exercise 3
  • Create a "screen saver" animation that runs until you move the mouse or kit a key.
• Detecting Where the Mouse is in Processing
• Exercise 4
  • Make your landscape interactive. Add a sunrise or another animation.

Keeping Track of Data

• Arrays in Processing
• Toggling Data in Processing
• Exercise 5
  • Using arrays, create a fading trail of circles that follow the mouse.
  • Add the ability to turn on and off the trail each time you click the mouse.

Expanding Processing

• Using External Libraries
  • OpenGL and GL Graphics
  • ToxicLibs
  • Video

Session 3: Arduino

In this session you will learn how to work with digital and analogue sensors that detect what is happening the real world.

Introducing Arduino

• Setting Up An Arduino
• Intro to the Arduino
• Exercise 1
  • Make the on-board LED turn on for 3 seconds then off for 5 seconds only once, then continuously turn on for 10 seconds and off for 7 seconds.
• How to Use a Breadboard
• Resistors
• LEDs and Arduino
• Exercise 2
  • Make an external LED turn on and off every 4 seconds continuously.

Working with Sensors

• Digital Sensors and Arduino
• Switches and Arduino
• Exercise 3
  • Turn on an LED when a switch is pressed and off when the switch is released.
  • Turn on and off an LED each time a switch is pressed and released.
• Analogue Sensors and Arduino
• Potentiometers and Arduino
• Exercise 4
  • Use a potentiometer to create a dimmer switch for an LED.
• Light Dependent Resistors and Arduino
• Exercise 5
  • Create a light-sensitive LED that dims as it gets dark.
• Piezos and Arduino
• Exercise 6
  • Make a sound meter with an LED and piezo.

Session 4: Putting It All Together

In this session we will combine what we've learned in previous sessions to make an audio and video installation that responds to physical actions by participants. You will have a chance to refine and finish the project that you have been developing each week.

Introduction to OSC

• What is OSC?
• Sending and Receiving OSC Data Using Processing
• Sending and Receiving OSC Data in Max
• Exercises
  • Send a message from Processing to Max and vice versa
  • Draw something on the screen in Processing relating to Max sound generators
  • Generate sounds in Max using some form interaction (e.g. mouse click) in Processing
  • Send a message to someone else's computer (by changing the IP address!)

Introduction to Serial

• Communicating with an Arduino Over Serial
• Communicating with Processing Over Serial
• Communicating with Max Over Serial
• Exercise 2
  • Make a theremin-like instrument with a photoresistor.
• Exercise 3
  • Draw an image that reacts to the ambient light level in the room.

• Exercise 4
  • Design and build an interactive system that uses physical sensors to trigger sounds in Max and visuals in Processing.

• This page was last modified on 17 March 2012, at 19:53.
• This page has been accessed 7,266 times.