代写COM3502程序、代做CS，Java程序语言、Python，c++编程代写

日期：2020-12-10 10:15

University of Sheffield

COM3502-4502-6502

Speech Processing

Programming Assignment

Contents

1 Overview (please read carefully) 2

1.1 Logistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Provided Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Hand-In Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Part-I: Voice Manipulation 4

2.1 Continuous Speech Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2 Filtering in the Time-Domain . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.3 Filtering in the Frequency-Domain . . . . . . . . . . . . . . . . . . . . . . 6

2.4 Low Frequency Oscillator . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.5 Amplitude Modulation: Tremolo . . . . . . . . . . . . . . . . . . . . . . . 9

2.6 Ring Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.7 Frequency Shifting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.8 Frequency Modulation: Vibrato . . . . . . . . . . . . . . . . . . . . . . . . 12

2.9 Time Delay Effects: Echo, Comb Filtering and Flanger . . . . . . . . . . . 12

2.10 Feedback: Reverberation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3 Part-II: Real-Time Voice Changer 15

3.1 General Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.2 COM4502-6502 Only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.3 Final Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1

1 Overview (please read carefully)

Before proceeding with this assignment you should have completed the ‘Pure Data’ (Pd)

Introductory Programming Exercise, and completed the quiz on Blackboard. If not, then

you are strongly advised to do so before moving on to the programming assignment described

below. In any event, it may be advisable to review the introductory exercise,

especially Sections 7 and 8 which cover working with real-time audio and speech.

Note: This programming assignment is worth 55% of the overall course mark.

This is an individual assignment.

You are permitted to re-use any of the Pd examples provided during the course.

1.1 Logistics

You are free to complete this assignment in your own time. However, feedback, advice

and guidance is available from the post-graduate teaching assistants via the appropriate

discussion board on Blackboard. Note that they will try to help you as much as possible,

but it is not their job to debug your code or provide solutions to the assignment itself.

Note: It will take some time to complete this assignment, so plan your work accordingly

over the coming weeks. Read these instructions carefully, and pay particular attention to

the marks associated with each component - especially the last.

Note: Please be aware that students registered on COM4502 and COM6502 have additional

tasks to perform. These are marked ‘COM4502-6502 Only’.

1.2 Provided Materials

In addition to these instructions, you have been provided with a .zip file containing a

number of items1

that you will need for this assignment. In particular, you have been

supplied with a LATEXtemplate - YourName.tex - which will you use to compile your

response sheet.

LATEX (pronounced [lA:tek] or [leItek]) is a free document preparation system for highquality

typesetting2

, and having a working knowledge of LATEX is a key skill for any

computer scientist3

. LATEX is very easy to learn4

, and there are many resources available on

the web, e.g. https://www.sharelatex.com/learn/Learn_LaTeX_in_30_minutes.

LATEX distributions are available for Linux, Mac OSX and Windows (see: https://www.

latex-project.org/get/). However, for this assignment, you might like to use an on-line

collaborative environment such as Overleaf (https://www.overleaf.com).

1The provided screenshot.jpg is simply a placeholder which you will replace with your own images.

2LATEX includes features designed for the production of technical and scientific documentation, so it

is the de-facto standard for the publication of scientific/technical papers and reports. Unlike WYSIWYG

(“What You See Is What You Get”) word processors (such as Microsoft Word or OpenOffice), LATEX

is based on plain-text source files (containing html-style markup) which are compiled into a typeset

document. This separation of ‘content’ from ‘style’ facilitates the production of very professional scientific

documents in terms of their consistency, readability and design. You can find out more about LATEX here:

https://www.latex-project.org.

3The Department strongly recommends that you consider using LATEX for your Dissertation, and Prof.

Moore has provided a template here: http://staffwww.dcs.shef.ac.uk/people/R.K.Moore/campus_

only/USFD_Academic-_Report_LaTeX-Template.zip.

4Students who took COM2009-3009 Robotics should be familiar with LATEX already.

2

? Step 0: Before you start working with LATEX, edit the filename of the provided .tex

file by replacing YourName with your name.

Your edited .tex file should compile to produce a .pdf document with your name on the

front cover, followed by your responses to a number of questions. You will be submitting

the .pdf file when you have completed the assignment, not the .tex file.

1.3 Hand-In Procedure

Once you have completed the assignment, you should submit a .zip file (via Blackboard)

containing your response sheet (in .pdf format) and the requested Pd source

files. Do not submit your LATEX source files. The .zip filename should be of the form

YourName.zip.

Please make sure that your name is shown correctly on the front page of your response

sheet (as explained in the previous section).

Standard departmental penalties apply for late hand-in5 and plagiarism6

.

Feedback (including provisional marks) will be provided via Blackboard within three working

weeks of the hand-in date.

The deadline for handing-in this assignment (via MOLE) is . . .

COM3502-4502: 17:00 Friday 18th December 2020

COM6502: 17:00 Friday 22nd January 2021

5

https://sites.google.com/sheffield.ac.uk/comughandbook/general-information/

assessment/late-submission

6

https://sites.google.com/sheffield.ac.uk/comughandbook/general-information/

assessment/unfair-means

3

2 Part-I: Voice Manipulation

As you saw in Lecture 1, speech processing has many applications across a wide variety

of market sectors. One area of particular interest is the creation of appropriate voices for

fictional characters in television, cinema, games and related areas of edutainment7

. Such

robot, alien or cartoon voice effects are often achieved by manipulating the speech of a

voice actor (either in real-time or in post-production8

) - a process that is referred to as

‘Voice FX’ or ‘Vocal FX’.

The aim of this assignment is to implement a range of speech processing algorithms for

real-time voice manipulation. The final outcome will be a ‘Voice Changer’ that can be

configured to perform a variety of different manipulations on your own voice. Part-I

involves creating the core components in Pd, and Part-II brings them together into a

single [VoiceChanger] application. Please work through the following sections in the

order given, providing responses the relevant questions as you come to them (by editing

your .tex file).

Note: In Part-II you will reuse many of the components you create in Part-I. Hence, you

are strongly advised to adopt professional working practices, e.g. saving versioned copies

of each piece of software that you develop.

Note: Some of the steps below require you to listen critically to sounds generated by

your software. It is thus advisable for you to use headphones and/or work in a quiet

environment. Take care not to annoy other people with your sound output.

2.1 Continuous Speech Input

Your first task is to program a Pd patch that can provide a continuous stream of real-time

speech that can be fed into the various manipulation techniques which follow.

? Step 1: Create the following patch . . .

Note: Recall from the introductory programming exercise that the audio output control

will appear when you create [output～] object.

7Wilson, S., & Moore, R. K. (2017). Robot, alien and cartoon voices: implications for speechenabled

systems. In 1st Int. Workshop on Vocal Interactivity in-and-between Humans, Animals and Robots

(VIHAR-2017). Skovde, Sweden.

8Rose, J. (2012). Audio Postproduction for Film and Video. Taylor & Francis.

4

? Step 2: Click on the bang at the top of the patch, increase the volume slider on

[output～] and verify that you can hear the speech9 on a continuous loop.

? Step 3: Now click on [wsprobe～]. This should open a separate patch that displays

the waveform and spectrum in real-time (as you have seen several times in the lectures).

Experiment with the various options provided in the [wsprobe～] GUI, and take particular

notice of the differences between voiced and voiceless sounds.

QUESTION 1 (worth up to 5 marks)

Provide a screenshot of [wsprobe～] for a typical voiced sound, and explain the features

in the waveform and spectrum that distinguish it from an unvoiced sound. Hint: use the

‘snapshot’ feature in [wsprobe～] to obtain a static display.

2.2 Filtering in the Time-Domain

One of the most basic manipulations in Voice FX is to ‘filter’ a signal in order to increase or

decrease energy at particular frequencies. The simplest types of filter are ‘low-pass’, ‘highpass’

and ‘band-pass’, and these may be implemented in Pd using the [lop～], [hip～]

and [bp～] objects respectively.

? Step 4: Modify your patch to include a low-pass filter with a ‘cut-off frequency’ controlled

by a horizontal slider covering the range 0 to 10,000 Hz as shown here10

. . .

? Step 5: Experiment with different values for the cut-off frequency, and observe the

consequences for the output sound as well as the waveform and spectrum displays.

9The provided speech.wav file is the same as the one used in the Lectures (and as provided in

com3502-4502-6502_pd-examples.zip available on MOLE).

10Recall that the slider’s range can be set by right-clicking and adjusting its ‘properties’.

5

QUESTION 2 (worth up to 5 marks)

Which sounds are most affected when the low-pass cut-off frequency is set to around 500

Hz - vowels or consonants - and why?

? Step 6: Edit the [lop～] object to become a [hip～] object, and again experiment

with different values for the cut-off frequency.

QUESTION 3 (worth up to 5 marks)

How is it that the speech is still quite intelligible when the high-pass cut-off frequency is

set to 10 kHz?

As you have seen, these simple low-pass and high-pass filters have only one control parameter

- their cut-off frequency. A simple band-pass filter has two parameters - its ‘centre

frequency’ and its ‘Q factor’. The Q of a band-pass filter is defined as Q = 1/B, where

B is the bandwidth (in Hz). Hence a high-Q filter has a narrow bandwidth and vice

versa.

? Step 7: Edit the [hip～] object to become a [bp～] object, and add another horizontal

slider covering the range 0 to 100 as shown here . . .

? Step 8: Experiment with different values for the centre frequency and Q.

2.3 Filtering in the Frequency-Domain

The [lop～], [hip～] and [bp～] objects operate directly on the speech waveform. Hence,

they are examples of ‘time-domain’ signal processing. An alternative to time-domain

processing is to process signals in the ‘frequency-domain’, i.e. by modifying the spectrum

rather than the waveform.

6

A classic example of filtering in the frequency-domain is a ‘graphic equaliser’. This is a

common component of a high-fidelity (HiFi) audio system, and it allows the spectrum to

be shaped by setting gains across a range of frequencies. You have been provided with a

Pd ‘abstraction’11 [GraphicEqualiser～] that can perform this function.

? Step 9: Remove your [bp～] object and its slider controls, and insert the provided

[GraphicEqualiser～]. Connect it between [readsf～] and [output～], and experiment

with different filter profiles by clicking on the graph (in ‘run mode’) and dragging the curve

into different shapes. Note how you can make low-pass, high-pass and band-pass filters

(and many more), simply by drawing the appropriate shapes, as shown here . . .

? Step 10: Open the [GraphicEqualiser～] abstraction to see how it works (by rightclicking

on the object, and selecting open). Take particular note of how the GUI objects

(the graph and reset button) are made to appear in the parent patch using the ‘graphon-parent’

option in the sub-patch properties12

.

QUESTION 4 (worth up to 5 marks)

COM3502-4502-6502: The [GraphicEqualiser～] object uses an FFT internally; what

does FFT stand for and what does an FFT do?

COM4502-6502 ONLY: What is a DFT and how is it different from an FFT?

? Step 11: Use [GraphicEqualiser～] to simulate the effect of a land-line telephone by

eliminating all energy below 300 Hz and above 3,400 Hz.

2.4 Low Frequency Oscillator

Many ‘Voice FX’ are achieved by modifying some characteristic of the speech using a low

frequency oscillator or ‘LFO’. LFOs typically have two controls: speed (which is specified

by the frequency in Hertz) and depth (which specifies the magnitude of the effect). Your

[VoiceChanger] will require several LFOs, so it makes sense to implement one as a Pd

‘abstraction’.

11Refer back to the introductory programming exercise if you have forgotten what an abstraction is.

12Note that you can also look inside [output～] or [wsprobe～] to see the same ‘graph-on-parent’

functionality.

7

? Step 12: Using the skills you acquired in the introductory programming exercise,

create a Pd abstraction [LFO～] that is capable of generating an audio sine/square wave

between 0 and 50 Hertz (e.g. using the [osc～] object). The speed and depth of the LFO

should be controllable by suitable sliders, and they should appear in the object using the

‘graph-on-parent’ approach.

? Step 13: Add an option to select either ‘sine’ or ‘square’ wave output using a Vradio

button. Hint: to obtain a square wave, multiply the output of the [osc～] object by a

very large number and pass the result through [clip～ -1 1] before being multiplied by

depth.

Additional features that will be useful later in the assignment are:

? [number] GUI objects for speed and depth

? a reset function

? external inputs for speed, depth and reset

? external outputs for speed and depth

? an ability to handle creation arguments for speed and depth (also linked to reset)

? an ability to reset the phase of the oscillator when speed = 0 in order to ensure

consistent output when initialising/resetting

Your resulting [LFO～] abstraction should look something like this (note the use of speed

= 5 and depth = 0.5 as creation arguments) . . .

Although your [LFO～] object outputs audio, you are unlikely to be able to hear it as the

frequency is so low. However, you can check that it is functioning correctly by connecting

it to [wsprobe～] and/or by using the appropriate number GUI object to wind the speed

up to an audible frequency.

? Step 14: Test your [LFO～] object by creating an [LFO～-help] object.

QUESTION 5 (worth up to 10 marks)

With speed = 50 and depth = 0.5, what are the minimum and maximum amplitudes of

your LFO output, and how do they vary with changes in these two settings? Also, please

provide two screenshots: (a) your [LFO～-help] object and (b) the internal structure of

your [LFO～] object.

8

2.5 Amplitude Modulation: Tremolo

‘Tremolo’ is one of the most basic voice manipulations that makes use of an LFO. In

this effect, the amplitude of a speech signal is modulated, i.e. the speech waveform is

multiplied by a variable gain that ranges between 0 and 1. Your LFO outputs an audio

signal between -depth and +depth. So, in order to modulate the amplitude of the speech

correctly, the output of the LFO has to be scaled appropriately - in this case, by adding

depth (e.g. using the audio math object [+～] connected to the depth output of your

LFO) and dividing the result by 2 (e.g. using the audio math object [*～ 0.5]).

? Step 15: Implement ‘tremolo’ using your [LFO～] object using the principles given

above. The resulting program should look something like this (note the use of a [sig～]

object to convert numerical data to audio data) . . .

? Step 16: Experiment with different settings for speed and depth. In particular, note

that a square wave with speed between 3 and 4 Hz (and depth = 1) has a very destructive

effect on the intelligibility of the output. This is because 3-4 Hz corresponds to the typical

syllabic rate of speech.

2.6 Ring Modulation

Another basic effect is to multiply the speech signal by the output of an LFO. This is

known as ‘ring modulation’.

? Step 17: Modify your implementation of ‘tremolo’ by removing the two scaling objects

([+～] and [*～ 0.5]) and connecting the audio output of your [LFO～] directly to the

[*～] object.

9

? Step 18: Experiment with different settings for speed and depth, and note how the

timbre of the resulting sound is subtly different from ‘tremolo’.

Note: In the BBC TV series Dr. Who, the voices of the alien Daleks13 are generated by

a ring modulator with an LFO set to around 30 Hz. The voice actors also spoke using a

stilted monotonic intonation in order to enhance the effect. You can try this yourself by

adding a ‘live speech input’ option to your code (i.e. using [adc～] and some means to

toggle between the prerecorded and live input)14

.

QUESTION 6 (worth up to 5 marks)

In your own words15

, why is this effect known as ‘ring modulation’?

2.7 Frequency Shifting

Many Vocal FX are the result of altering the frequencies present, e.g. changing the pitch

of a voice. There are many algorithms for frequency shifting, but you have already implemented

an approximate solution with your ring modulator.

? Step 19: Use the number object in your [LFO～] GUI to wind up the speed of the

LFO in your ring modulator to around 300 Hz. If all is working correctly, as you raise the

modulation frequency, you will hear that the frequencies in the speech seem to be both

increasing and decreasing. Indeed, if you listen carefully at a fixed speed of around 300

Hz, you may be able to detect two voices speaking simultaneously at different frequencies.

The reason for this result is that multiplying one signal by another - known as ‘heterodyning’

- produces an output which consists of the sums and differences of the frequencies

of the two input signals. This means that the frequencies in the original speech are both

raised and lowered by an amount corresponding to the speed of the LFO - hence the

strange mixture of higher pitched and lower pitched voices in the output.

The sum and difference components are known as the upper and lower ‘sidebands’. So, to

avoid the distortion caused by having both sidebands present in the output, we need to

remove one of them, i.e. we only want one sideband. This can be achieved by creating two

versions of the original signal that differ in phase by 90?

, then modulating the two versions

with two heterodynes that also differ in phase by 90?

. These phase differences are such

that, when the two out-of-phase heterodyned signals are subtracted, the lower sideband

is cancelled out. This process is known as ‘single-sideband modulation’ or ‘SSB’16

.

QUESTION 7 (worth up to 5 marks)

Why is SSB commonly used in long-distance radio voice communications?

Splitting a signal into two with a 90? phase difference between them can be achieved using

the ‘Hilbert Transform’. Hence the following patch shows the Pd object [hilbert～] being

used to create a frequency shifter based on SSB modulation . . .

13https://en.wikipedia.org/wiki/Dalek

14You will need this feature later, in any case.

15I.e. do not plagiarise from Wikipedia.

16SSB is commonly used in long-distance radio voice communications, especially by amateur radio

enthusiasts.

10

? Step 20: Implement the above patch, and experiment with different values for the

frequency shift. Note that the frequency-shifted output now sounds much cleaner than

the output of the ring modulator (i.e. only one voice). Also note that the pitch of the

voice can be shifted down as well as up.

QUESTION 8 (worth up to 5 marks)

COM3502-4502-6502: Why can the voice be shifted up in frequency much further than

it can be shifted down in frequency before it becomes severely distorted? Hint: look at

[wsprobe～].

COM4502-6502 ONLY: Your frequency shifter changes all the frequencies present in an

input signal. How might it be possible to change the pitch of a voice without altering the

formant frequencies?

A classic ‘robotic’ voice can be achieved by simply adding frequency-shifted speech back

to the unprocessed original. This effect is known as ‘harmony’. However, rather than

simply adding the signals in equal amounts, your final [VoiceChanger] will benefit from

a more general purpose approach.

? Step 21: Implement a ‘mixer’ that adds the original speech with the manipulated

speech in different proportions. Use a slider that has 100% original at one end, 100%

manipulated at the other end and 50-50 in the middle.

? Step 22: With your mixer at the 50-50 setting, experiment with different frequency

shifts in order to produce the best robotic sounding output.

11

2.8 Frequency Modulation: Vibrato

Now that you have the ability to shift the frequencies in a speech signal, it is very easy to

implement another common voice manipulation technique - ‘vibrato’. All that is required

is for the frequency shifter to be controlled by the output of an LFO.

? Step 23: Implement ‘vibrato’ by connecting an LFO to your frequency shifter, and

experiment with different values for speed and depth. Note that the LFO output will

need to be scaled to provide an appropriate frequency shift range and then added to the

output of the frequency shift slider.

Note: With speech.wav as input, it is possible to simulate the voice of Gollum from the

Lord of the Rings by setting LFO = sine wave, speed = 50 Hz and depth = 350.

2.9 Time Delay Effects: Echo, Comb Filtering and Flanger

Many interesting voice FX can be achieved by delaying the signal and recombining it

with itself. [Pd] provides the possibility of writing to and reading from audio ‘delay

lines’ using the objects [delwrite～] and [delread～]. However, for full flexibility in

your [VoiceChanger] application, you need to be able to vary the time at which audio

data is read out of the delay line. So, rather than using [delread～] it is better to use

[vd～].

? Step 24: Implement the following patch17 with the ‘delay’ slider’s properties set to

operate on a log scale from 1 to 1000 msecs . . .

17Of course, you may choose to replace the [+～] with the mixer you implemented earlier.

12

? Step 25: Experiment with various values for the delay, and note the different effects

you can achieve with delays (a) below 20 msecs, (b) between 20 and 100 msecs, and (c)

above 100 msecs.

You should observe that, with delays below 20 msec, the signals combine to create a subtle

‘phasing’ effect. This is known as ‘comb filtering’ as the signal is effectively interfering

with itself, and frequency components corresponding to multiples of the delay time are

enhanced or cancelled out (due to ‘superposition’). Delays between 20 and 100 msecs give

the effect of the voice being in a reverberant room. Delays above 100 msecs sound like

distant echoes.

Finally, in this section, it is possible to use an LFO to vary the delay. The resulting effect

is known as a ‘flanger’.

? Step 26: Add an LFO to your ‘delay’ patch to create a ‘flanger’, and experiment with

different settings. Note that you will need to scale the output of the LFO, and you will

get different effects depending on whether the delayed signal is mixed with the original or

not.

Note: Using ‘flanger’ it is possible to achieve an underwater effect by setting LFO = sine

wave, speed = 10 Hz, depth = 0.4. delay = 5 msecs and mix = 100% processed.

2.10 Feedback: Reverberation

The previous section noted that an echo occurring between 20 and 100 msecs after the

original sounds like reverberation. However, in a real environment there are many echoes

caused by multiple reflections from the different surfaces present, and a signal might

bounce around a very reverberant environment many times. A simple method for achieving

such an effect is to feed a proportion of the output signal back to the input. This is known

as ‘feedback’.

? Step 27: Modify your delay patch as follows, and experiment with different values for

delay and feedback . . .

13

QUESTION 9 (worth up to 5 marks)

In a practical system, why is it important to keep the feedback gain less than 1?

Note: A short delay with a lot of feedback can give a metallic sound to a voice, e.g. C3PO

from Star Wars. Try delay = 10 msecs, feedback = 0.9 and mix = 90% processed.

Clearly feedback could be applied to any/all of the effects that you have implemented so

far, thereby increasing the range of possible voice effects considerably - as you will discover

in Part-II.

14

3 Part-II: Real-Time Voice Changer

3.1 General Requirements

The final part of this assignment is to compile all of the different components you developed

in Part-I into a single [VoiceChanger] application. The application should allow

‘live’ speech input in addition to the ability to select a particular prerecorded file (i.e. no

longer hard-wired for speech.wav). The GUI should be well thought out, easy to use and

attractive, and it should not only allow the different effects to be controlled, but also allow

them to be connected to each other in a logical sequence. You should also provide innovations,

for example preset effects (using Vradio buttons to select particular combinations

of settings).

3.2 COM4502-6502 Only

It is possible to convert a single voice into multiple voices by combining the outputs

of parallel manipulations, especially different frequency shifts. This effect is known as

‘chorus’. Your objective is to add ‘chorus’ to your [VoiceChanger] application by careful

use of abstractions and a method for selecting one or more output voices.

3.3 Final Steps

? Step 28: Implement [VoiceChanger] according to the requirements stated above.

QUESTION 10 (worth up to 50 marks18)

Please provide a short19 description of the operation of your [VoiceChanger] application,

together with a screenshot of your final GUI.

Congratulations, you have almost completed this programming assignment. All that is

left now is to package up your code and submit it for marking.

? Step 29: Create a .zip file of the form ‘YourName.zip’ containing . . .

? your response sheet in .pdf format with your name on the front cover and containing

your responses to all of the questions; and

? your final [VoiceChanger] code, together with any necessary abstractions (such as

[LFO～], [LFO～-help] etc.).

Important: For marking, we expect your code to work ‘out of the box’.

Note: Do not include your .tex file (and associated image files) or your work-in-progress

code examples.

? Step 30: Hand-in your .zip file in accordance with the instructions at the beginning

of this document.

That’s it - you’re done!

1825 for functionality, 15 for design/layout, 5 for Pd features, 5 for innovations

19no more than 500 words

15