Christmas Voice – Part 1

Once again it’s Christmas at Styling Android towers. I’m based in the UK and here many companies shut down over the festive period, so I like to do something a little more light-hearted. If you’re celebrating a different religious festival at this time of year then I wish you a happy one; or if you’re celebrating absolutely nothing, then I still hope it’s a happy time. But for those who do celebrate Christmas, today I have released Christmas Voice to Google Play. It is a voice changer app which allows you sound like either Santa Clause or, if you prefer, an Elf. The app is completely free, with no adverts; and it’s also open-source (this link is at the end of the article). The app is only available for devices running Marshmallow and later (API23+) for reasons which will become obvious as we look at the technique used to perform the audio transformation. In this short series of articles we’ll take a look at how it works.

Before we dive in, I’m not going to give a full explanation of all of the app – much of it should be fairly obvious – the core function of the app is recording an audio clip and then playing it back with some modifications to the audio to create the desired effect. It is this functionality that we’ll focus upon.

The first thing we’ll look at is how we capture the audio. We’re going to use AudioRecord to capture the audio from the device microphone, and write it to a file. The AudioRecord instance is created by the MediaToolsProvider class:

public class MediaToolsProvider {

    AudioRecord getAudioRecord() {
        AudioFormat audioFormat = getAudioFormat(AudioFormat.CHANNEL_IN_MONO);
        int bufferSize = getInputBufferSize(audioFormat);
        return new AudioRecord.Builder()
                .setAudioSource(MediaRecorder.AudioSource.MIC)
                .setAudioFormat(audioFormat)
                .setBufferSizeInBytes(bufferSize)
                .build();
    }

    private AudioFormat getAudioFormat(int channelMask) {
        int sampleRate = AudioTrack.getNativeOutputSampleRate(AudioManager.STREAM_MUSIC);
        return new AudioFormat.Builder()
                    .setSampleRate(sampleRate)
                    .setChannelMask(channelMask)
                    .setEncoding(AudioFormat.ENCODING_DEFAULT)
                    .build();
    }

    private int getInputBufferSize(AudioFormat audioFormat) {
        return AudioRecord.getMinBufferSize(audioFormat.getSampleRate(),
                                            audioFormat.getChannelCount(),
                                            audioFormat.getEncoding());
    }
    .
    .
    .
}

The AudioFormat object specifies how the audio will actually be captured and it is important to tie this up with how we intend to play the audio back later on. We must specify the sample rate (which we determine by querying the device to determine the native output sample rate); the channel configuration (we specify CHANNEL_IN_MONO as we’re recording from a mono microphone); and the how the audio will be encoded (once again we’ll use the device default as it less likely that we’ll encounter hardware issues if we use the audio hardware’s default encoding).

Next we need to determine the size of the input buffer – which is a temporary buffer which will be used during the recording itself. We determine this by calling AudioRecord.getMinBufferSize() with values from the AudioFormat object.

Now we can create the AudioRecord object by specifying the audio source – in this case we specify the microphone, plus the AudioFormat and buffer size that we’ve already determined.

We now have an AudioRecord, but we will not be able to do anything with this unless we have the RECORD_AUDIO permission because we have specified that we need to record from the microphone. The code for doing that is in the source, but I won’t bother covering it here.

Now that we have the AudioRecord object we can record the audio from it which is controlled by the AudioRecorder class:

class AudioRecorder implements Recorder {
    private final AudioRecord audioRecord;
    private final File file;

    private Thread recorderThread;
    private AudioRecorderTask recorderTask;

    AudioRecorder(AudioRecord audioRecord, File file) {
        this.audioRecord = audioRecord;
        this.file = file;
    }

    @Override
    public boolean isRecording() {
        return recorderThread != null && recorderThread.isAlive();
    }

    @Override
    public boolean hasRecording() {
        return file.exists();
    }

    @Override
    public void startRecording() {
        recorderTask = new AudioRecorderTask(audioRecord, file);
        recorderThread = new Thread(recorderTask);
        recorderThread.start();
        audioRecord.startRecording();
    }

    @Override
    public void stopRecording() {
        audioRecord.stop();
        audioRecord.release();
        recorderThread = null;
        recorderTask = null;
    }
}

There isn’t really that much to explain here. The startRecording() method creates AudioRecorderTask which performs the input from the AudioRecord object on a background thread. Finally we call AudioRecord#startRecording() to begin the actually recording.

Stopping the recording is simply a matter of stopping and releasing everything. It is important to call release() on the AudioTrack object to ensure that everything gets cleaned up.

All that’s left is the AudioRecorderTask:

class AudioRecorderTask implements Runnable {
    private static final int BUFFER_SIZE = 1024;

    private final AudioRecord audioRecord;
    private final File outputFile;

    AudioRecorderTask(AudioRecord audioRecord, File outputFile) {
        this.audioRecord = audioRecord;
        this.outputFile = outputFile;
    }

    @Override
    public void run() {
        OutputStream outputStream = getOutputStream();
        if (outputStream == null) {
            return;
        }
        byte[] buffer = new byte[BUFFER_SIZE];
        int read = audioRecord.read(buffer, 0, BUFFER_SIZE);
        while (read > 0) {
            try {
                outputStream.write(buffer, 0, read);
            } catch (IOException e) {
                Timber.e(e, "Error writing audio file");
            }
            read = audioRecord.read(buffer, 0, BUFFER_SIZE);
        }
        try {
            outputStream.close();
        } catch (IOException e) {
            Timber.e(e, "Error closing audio file");
        }
    }

    private OutputStream getOutputStream() {
        OutputStream outputStream;
        try {
            outputStream = new FileOutputStream(outputFile);
        } catch (FileNotFoundException e) {
            Timber.e(e, "Error opening audio file for writing");
            return null;
        }
        return outputStream;
    }
}

Once again, there’s nothing difficult here, we first create an OutputStream for the File which is where we’ll write the audio data. We then just keep reading from the AudioRecord object and writing that data to the OutputStream until we run out of data (which will happen once the AudioRecord is stopped, and we have read any remaining data. Finally we clean everything up. It is worth pointing out that recording will be stopped by external forces such as either the user hitting a stop record button, or a timer expiring – so we don’t need to perform any clean up of the AudioRecord instance here, that will have been done already and it is that which will result in the buffer emptying.

So now we have the audio stored to a file. In the concluding article in this series we’ll take a look at how we play the audio back, and apply the audio transformations.

Many thanks to my testers: Darren, Ben, Jenny, Roberto, Seb, Dario, Donn, Naresh, Gyuri, Kenton, Wiebe, Mike, & Danny. Additional thanks to Seb for proof-reading – any remaining typos are all mine, but there would have been many more without Seb’s keen eye!

The source code for Christmas Voice is available here.

© 2016, Mark Allison. All rights reserved.

Copyright © 2016 Styling Android. All Rights Reserved.
Information about how to reuse or republish this work may be available at http://blog.stylingandroid.com/license-information.