Sound and music¶

Play sounds from your script, and draw what the music does.

What you get¶

class Bell
  def on_button(btn)
    if btn == "select" && !sound.playing()
      sound.play({'rtttl': 'bell:d=4,o=5,b=100:e,c'})
    end
  end

  def draw()
    clear()
    text(1, 6, "Bell", 0xFFFFFF)
  end
end

return Bell()

Press select while the app is shown, and the clock plays a short melody. sound.playing() keeps a quick double press from playing it twice.

How it behaves¶

• sound.play() returns at once, and the sound plays while your script goes on.
• true means the clock took the sound, not that you hear it. A name that is not stored plays nothing, and the volume may be 0. false means this clock can play none of it. What the answer means lists every case.
• Your script plays three kinds of sound at once: a single sound on top, effects over the music, and music underneath everything. See Music and effects.
• An alert is never cut off by a script. The sound of a notification plays over your script's sounds.
• music turns sound into numbers you can draw. Read them in draw(): the readings for playback match the moment you hear the sound.
• Start sounds in a button handler or in loop(), never in draw(), which runs about 40 times a second.

What to play¶

sound.play(x) takes a sound, the same one /api/v1/audio/play takes. In Berry it is a string, a map or a list:

sound.play("doorbell")                           # a stored MP3 or melody, by name
sound.play({'rtttl': 'beep:d=16,o=6,b=200:c'})   # a melody in the call
sound.play({'file': 'siren', 'loop': true})      # repeats until you stop it
sound.play([{'speech': 'Door open'}, 'doorbell'])  # speech, else the doorbell

A map has exactly one of these keys. The key says what plays:

Next to the key, a map may have:

• 'loop': true to repeat the sound until it is stopped.
• 'nextBar': true, only with 'song' and 'loop': true. See Songs and synth effects.

A plain name is short for {'file': name}. A list holds 1 to 4 sounds, and the clock plays the first one it can play.

'station' starts the radio, as the web UI does. It is not allowed in a list, and sound.stop() does not stop it.

One sound after another¶

sound.playing() lets you wait for one sound to finish before you start the next. A button that plays a sound needs this, or a quick double press plays it twice. See What you get.

Play sounds one after another in loop(), never in draw():

  var queue

  def init()
    self.queue = ["chime", "alarm"]     # sounds waiting to play
  end

  def loop()
    if size(self.queue) > 0 && !sound.playing()
      sound.play(self.queue.pop(0))
    end
  end

Use notify() when the sound belongs to an event that should also interrupt the rotation and show something. Use sound when you only want the sound. A notify() sound is an alert, so it plays over your script's sounds. See Notifications.

Different clocks¶

Clocks have different sound hardware. Ask sound.can() before you play something only some clocks have:

  def on_button(btn)
    if btn == "select" && !sound.playing()
      if sound.can('mp3') sound.play("doorbell")
      else sound.play({'rtttl': 'bell:d=4,o=5,b=100:e,c'}) end
    end
  end

A list does the same in one call. The clock plays the first entry it can play:

sound.play(["doorbell", {'rtttl': 'bell:d=4,o=5,b=100:e,c'}])

sound.can() answers these keys, among others. They are the audio flags of GET /api/v1/capabilities:

A script that cannot work without one of them says so in its header, for example # @needs audio.effect. See Sharing.

Music and effects¶

Your script plays three kinds of sound at once. This is what a game needs:

  def on_show()
    sound.play({'file': 'theme', 'loop': true})
  end
  def on_hide()
    sound.stop()
  end
  def on_button(btn)
    if btn == "select" sound.effect("jump") end
  end

• Stored MP3s and song text play as effects. Any other sound, an address for example, plays like sound.play().
• An effect plays once, even with 'loop': true.
• Up to four MP3 effects and four song effects play together. A fifth cuts off the oldest. An effect plays for at most 10 seconds.
• Asking for the music that already plays changes nothing. Other music replaces it.
• While a single sound plays, the music gets quieter. While an alert plays, the effects get quieter too.
• Your script's sounds stop when you call sound.stop(), and when the script is deleted, saved again or stops with an error. Stop them in on_hide() when they belong to what your app shows.
• Your script's sounds pause the radio. It comes back once they have ended, the music included.
• A radio station that starts ends the music and the effects. A single sound that plays finishes first.
• Home Assistant Voice silences everything while it listens and answers. The music then starts again from the beginning. Music you started or stopped meanwhile counts. Effects do not come back.
• Everything your script plays is at the Apps volume of the mixer.

Clocks without effects play one sound at a time there. A script meant for every clock checks sound.can('effect') or sound.can('song') before it uses sound.effect() or sound.beat().

MP3s from the internet¶

'file' also takes a web address. The script does not need to bring the file along.

  def on_show()
    if sound.can('url')
      sound.play({'file': 'https://example.com/rain.mp3', 'loop': true})
    end
  end

• The clock downloads the whole file first, then plays it. That takes a moment.
• A file can be up to 4 MB, and the clock needs enough free memory for it.
• Asking again for the same address as music changes nothing. The music keeps playing.
• true only means the request was accepted. If the file cannot be downloaded, nothing plays, and sound.playing() stays false.

Speech¶

A clock with a voice reads English text aloud:

# @needs audio.speech

class TalkingClock
  def on_button(btn)
    if btn == "select"
      sound.play({'speech': "It is " + str(hour()) + ":" + format("%02d", minute())})
    end
  end

  def draw()
    clear()
    text(0, 6, format("%02d:%02d", hour(), minute()), 0xFFFFFF)
  end
end

return TalkingClock()

• The text is a string. Build it with str() and format(), as above.
• The clock reads it as described in Speech: up to 512 bytes, numbers and times as words, emoji skipped.
• Speech is a single sound like an MP3. The next sound replaces it, sound.stop() stops it, and sound.playing() is true while it speaks.
• On a clock without a voice, sound.play() returns false. Give a list to play something else there: sound.play([{'speech': 'Door open'}, 'doorbell']).
• Empty text, or text longer than 512 bytes, raises a value_error. On a clock with a voice, so does text with nothing to read, for example '...'.

Songs and synth effects¶

Your clock has a synthesizer, sound.can('song'). It plays music written as song text: instruments and notes in a string, with no file to upload. sound.play() with 'loop': true plays a song as music, sound.effect() plays one over it, and sound.beat() tells your picture where the music is. Song calls lists every way to start one.

# @name   Pulse
# @needs  audio.song

class Pulse
  var tune

  def init()
    self.tune = "bpm 100\n"
                "inst pad  wave=saw unison=12 attack=200 sustain=60 release=400 cutoff=1500 volume=65\n"
                "inst kick wave=sine note=c2 pitch=24/25 attack=0 decay=200 sustain=0 volume=200\n"
                "pad:  [a3 c4 e]:16 | [f3 a c4]:16 |\n"
                "kick: (%x...x...x...x...)2"
  end

  def on_show()
    sound.play({'song': self.tune, 'loop': true})   # the music starts with the app
  end
  def on_hide()
    sound.stop()                                    # and stops with it
  end
  def on_button(btn)
    if btn == "select"
      sound.effect({'song': "inst blip wave=sine attack=0 decay=60 sustain=0 volume=200\nblip: c6:1"})
    end
  end

  def draw()
    clear()
    var beat = sound.beat()
    if beat == nil return end
    var r = int(6 * (1 - beat % 1))     # big on the beat, shrinking until the next
    circle_fill(width() / 2, height() / 2, r, 0xFF4000)
  end
end

return Pulse()

Berry joins string literals that follow each other, even across line breaks and comments. So you can write a song one statement per line, as above. Every piece except the last ends in \n, the line break between two statements.

• On a clock without a synthesizer, sound.play returns false.
• A mistake in the text raises a value_error with the reason, line and column, for example song: unexpected 'x' (line 2, column 7). Text you build while the script runs belongs in a try.
• A beat is a quarter note. sound.beat() % 1 runs from 0 on each beat to almost 1 just before the next, and int(sound.beat()) % 4 counts the beats of a 4/4 bar. When the song loops, the count goes back to the first beat of its loop bar. A song started with 'nextBar': true counts from 0 once it takes over.
• sound.beat() matches what comes out of the speaker, like the music readings. Read it in draw() and the picture keeps time with the sound. A song with loop off in its text plays once, and sound.beat() returns nil once it has ended.
• Song effects count for sound.playing(). The music does not.
• Up to four song effects play at once. A fifth cuts off the oldest.

Sounds for your script¶

A script can bring its own sounds: MP3 files in a folder named after the script, next to it.

/SCRIPTS/Racer.ax           the script
/SCRIPTS/Racer/boost.mp3    its sounds
/SCRIPTS/Racer/theme.mp3

The script plays them by name, like any MP3:

  def on_show()
    sound.play({'file': 'theme', 'loop': true})   # /SCRIPTS/Racer/theme.mp3
  end
  def on_button(btn)
    if btn == "select" sound.effect("boost") end
  end

A plain name is looked up in this order:

1. the script's own folder, /SCRIPTS/Racer/boost.mp3;
2. the shared MP3s, /MP3/boost.mp3;
3. the melodies, /MELODIES/boost.txt.

sound.play(), sound.effect() and the sound of a notify() look this way. Anywhere else, in a request or in another script, the sound is "Racer/boost".

A module has no sounds of its own. Its code plays from the folder of the app that imports it, so that app brings the sounds.

Add sounds to a script:

• From the AWTRIX Hub, sounds come with the script. This works from its Hub page, and in the web UI when it updates a script or installs one that another script needs. A clock that cannot play MP3s gets the script without them.
• By hand (a shared .ax file carries no sounds): open the script on the Scripts tab and press the note button in the editor bar. It opens the script's group on the Audio tab, where you upload, play and delete its sounds. See The web UI → MP3s.

Deleting the script asks about its sounds. Delete with sounds removes them. Keep sounds leaves them on the clock, and a script installed again under the same name uses them. Saving a new version keeps them. Renaming the script in the web UI moves them to the new name.

The files follow the rules for every MP3 on AWTRIX:

• The file name is what the script plays: 1 to 32 characters of A-Z, a-z, 0-9, _ and -, plus .mp3. boost.mp3 is played as "boost".
• The sound must be an MP3, mono or stereo, at 8 to 48 kHz. A file that is not an MP3 is refused on upload or stays silent.
• They share storage with everything else on AWTRIX, so keep them short.
• A script's sound may have the same name as a melody or a shared MP3. Its own sound wins.

Listing, uploading and deleting over the API: Script sounds.

Music¶

music turns sound into numbers you can draw. It analyses playback or the microphone. Choose under System → Audio → Music source. Automatic uses playback while something plays, otherwise the microphone. Playback and Microphone pick one source.

The readings for playback match the moment you hear the sound, so picture and sound stay in step. music.bands() answers how loud each range of notes is, from bass to treble, music.level() how loud it is, and music.beat() when a beat lands. music.station() and music.title() name what the radio plays. Music calls lists every answer.

None of them ever returns nil. When there is no input, they return zeros, false and "".

The radio does not put the station or the song on the display by itself. A script that shows them builds the line in loop(). music.title() changes whenever the station announces a new song:

def init()
  self.line = ""
end

def loop()
  self.line = music.station() + ": " + music.title()
end

def draw()
  clear()
  scroll_text(self.line, 0xFFFFFF)
end

def should_show()
  return music.title() != ""
end

music.playing() reports playback only, whatever input is selected. It does not detect voices or room sound. So for a microphone visualizer, do not hide the app with a should_show() based on music.playing().

A spectrum display takes one line. bar_chart() with autoscale off draws a fixed 0-to-8 range, so ask for the bands with max 8:

def draw()
  bar_chart(music.bands(16, 8), "Rainbow", false)
end

The classic look adds a peak dot per bar that hangs for a moment and then falls. This one also skips its turn while nothing plays:

class Spectrum
  var peaks
  var hold

  def init()
    self.peaks = []
    self.hold = []
    for i: 0..15
      self.peaks.push(0)
      self.hold.push(0)
    end
  end

  def should_show()
    return music.playing()
  end

  def draw()
    var bands = music.bands(16, 8)
    var bar_w = (width() - 15) / 16
    bar_chart(bands, "Rainbow", false)
    for i: 0..15
      if bands[i] >= self.peaks[i]
        self.peaks[i] = bands[i]
        self.hold[i] = 12
      elif self.hold[i] > 0
        self.hold[i] -= 1
      elif self.peaks[i] > 0
        self.peaks[i] -= 1
      end
      if self.peaks[i] > 0
        pixel(i * (bar_w + 1), height() - 1 - self.peaks[i], 0xFFFFFF)
      end
    end
  end
end

return Spectrum()

music.beat() is for a pulse, a circle that flares on every beat and shrinks again:

  var glow

  def init()
    self.glow = 0
  end

  def draw()
    if music.beat() self.glow = 4 end
    if self.glow > 0
      circle_fill(width() / 2, height() / 2, self.glow, hsv(300, 100, 100))
      self.glow -= 1
    end
  end

• The levels adjust themselves: a quiet track fills the display just like a loud one, and the volume setting does not change the picture.
• Read music.beat() in draw(), never in loop(). loop() runs once a second and would miss most beats.
• music.playing() turns true as soon as playback starts. The first numbers follow a fraction of a second later.

Hearing a note¶

music.pitch() tells which note is sounding: one that someone sings, hums, whistles or plays into the microphone, or one that the speaker plays. It answers the note's frequency in Hz, from 70 Hz, a low bass voice, to 1600 Hz, a high whistle.

It listens to the same Music source as the other readings. With Automatic, a sound your script plays is heard instead of the singer, so a singing game stays quiet while someone sings. Declare # @needs microphone for a script that cannot work without the microphone.

It answers 0.0:

• for silence, noise, several notes at once, or a note outside 70 to 1600 Hz;
• while the voice assistant is listening.

Call it in draw(), not in loop(). The first reading comes a fraction of a second after the first call.

A note number follows from the frequency: 69 is A4 at 440 Hz, and each semitone adds one. This app shows the note name, such as A4:

import math

class Tuner
  var names, note, label

  def init()
    self.names = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B']
    self.note = nil
    self.label = ""
  end

  def draw()
    var hz = music.pitch()
    if hz <= 0 return end
    var note = int(69 + 12 * math.log(hz / 440) / math.log(2) + 0.5)
    if note != self.note                # build the name only when the note changes
      self.note = note
      self.label = self.names[note % 12] + str(note / 12 - 1)
    end
    text(0, 6, self.label, 0x00FF00)
  end
end

return Tuner()

Good to know¶

• A quick double press plays a sound twice. Check !sound.playing() before you play, as in What you get.
• Start sounds in a button handler or in loop(), never in draw(). draw() runs about 40 times a second.
• A sound that repeats keeps playing when your app leaves. Stop it in on_hide() when it belongs to what your app shows.
• A mistake in song text raises a value_error. Put song text you build while the script runs into a try.
• Read music.beat() in draw(), never in loop(). loop() runs once a second and misses most beats.

Details¶

Calls¶

What the answer means¶

true means the clock took the request, and the sound starts right away. true does not mean you hear it:

• A name that is not stored plays nothing.
• While an alert plays, for example the sound of a notification, a single sound from your script is not played. An alert is never cut off by a script.
• The volume may be 0.

false means this clock cannot play any of it, for example {'track': 3}, because your clock has no DFPlayer. It never stops your script.

A mistake in the sound raises a value_error at the call, with the key and the reason:

sound.play({'rtttl': 'd=4,o=5,b=120:c,e,g'})
# value_error: rtttl: missing ':' (at offset 19)

A map with two sound keys, an unknown key, a bad name or a track out of range raise the same way. The reasons are listed in When it goes wrong.

Song calls¶

Music calls¶

Every sound limit is in Limits.

Related¶

• Sound: stored sounds, melodies, the volume and what each clock can play
• Song text: instruments and notes in a string
• Internet radio: the stations music reads
• Controlling the device: a sound together with a notification