COMP0090代写、代写c++,python设计程序

日期：2022-08-12 08:26

COMP0090 Coursework 2 (originally the “group work”) -

Multi-task learning (LSA)

1 Introduction

Multi-task learning (MTL) is a field of machine learning which focuses on learning multiple tasks either

simultaneously or sequentially [1, 2, 3]. The aim of MTL is to leverage information from multiple tasks

in order to improve performance for either all the tasks being learnt, or for a single separate ‘target task’.

In the case that target task(s) exist, auxiliary task(s) may be learnt prior to or simultaneously with the

target task(s). Under this definition, ‘transfer learning’ may be considered as a special case of MTL with

one auxiliary task and one target task, where the learning is sequential rather than simultaneous.

2 The project

In this project you will implement and evaluate a MTL framework for a target task of animal segmen-

tation. Learning auxiliary task(s) simultaneously with or prior to learning this target task may improve

segmentation performance. The choice and number of auxiliary tasks, as well as the decision of whether

learning is simultaneous or sequential, can all impact the target task performance in MTL. You will be

implementing a MTL framework and evaluating this with respect to a non-MTL baseline as well as to a

MTL framework that omits certain components (ablation study).

The goal of this project is to assess your ability to reason about design choices in deep learning ex-

periments, to implement deep learning frameworks, and to evaluate the developed deep learning models,

akin to conducting a research study. Your findings are to be compiled in a report which will follow a

similar structure to a research paper.

N.B. This assessment component is originally a group project. This late summer assessment (LSA)

has been adjusted and can also be completed by individuals, without the peer assessment.

2.1 The data set

The Oxford-IIIT Pet Data-set will be used in this project where a pre-processed version has been made

available in an open-source repository. The pre-processed data contains 3686 samples with three kinds

of task labels available for each sample: 1) binary classification of dog vs cat (‘binary.h5’); 2) animal

bounding boxes (‘bboxes.h5’); 3) animal segmentation masks (‘masks.h5’). In addition to these tasks,

other tasks that utilise self-supervision such as self-reconstruction of the images using auto-encoding,

predicting random rotations for images, predicting random flips for images etc. may be used, if suitable.

2.2 The “minimum required project”

The so-called minimum required project (MRP) (80%) for this group work consists of the following

points, which are expected to be found in your submission. Some of these points are discussed in more

detail to get you started.

1. Deciding and justifying which MTL formulation to use

2. Implementing a data loading pipeline

3. Implementing a MTL framework

4. Implementing a baseline network and an ablated version of MTL

5. Experimenting for network comparison

1

6. Describing implemented methods and conducted experiments

7. Summarising obtained results and drawing conclusions

2.2.1 Deciding and justifying which MTL formulation to use

In this project we only focus on a MTL formulation with a single target task with potentially multiple

(≥1) auxiliary tasks. It should be noted that formulations with multiple target tasks or all tasks being

the target task are also valid MTL formulations. As introduced above, MTL may be formulated in one

of two ways: 1) simultaneous learning: auxiliary task(s) are learnt together with the target task; 2)

sequential/ transfer learning: auxiliary task(s) are learnt prior to the target task. In this project you

must decide which formulation you want to use as well as present a brief survey of both kinds of MTL

formulations (by citing previous work). The choice must be justified in the report.

Hint: some keywords that you can search for, are: 1) ‘multi-task leanring’; 2) ‘transfer learning’; 3)

‘parameter sharing’; 4) ‘deep learning fine-tuning’; 5) ‘shared weights’.

2.2.2 Implementing a data loading pipeline

For this working point you must implement one data generator/ loader to sequentially load batches of

data from h5 files, for the purpose of your project. For the loading pipeline in MTL, the output must be

compatible with the deep learning framework you use for this project (either TensorFlow or PyTorch).

Hint: some keywords that you can search for, are: 1) ‘multi-task learning’; 2) ‘multi-output models’;

3) ‘python yield statement’; 4) ‘python generator’; 5) ‘tensorflow data generator’; 6) ‘pytorch data loader’.

1 import h5py

2 import os

3 import numpy as np

4

5 # number of samples to load from h5 file

6 num_samples = 256

7

8 # define the path for the h5 files (in this case we use the train set)

9 h5_save_path = r’COMP0090_CW/train ’

10

11 img_path = r’images.h5’

12 mask_path = r’masks.h5’

13 bbox_path = r’bboxes.h5’

14 bin_path = r’binary.h5’

15

16 total_samples = 2210

17

18 # generate random indices which are to be sampled from the dataset

19 inds_to_sample = np.sort(np.random.choice(total_samples , size=num_samples , replace=False

))

20

21 # define a function to load only samples at the specified indices

22 def load_data_from_h5(path , inds_to_sample):

23 with h5py.File(path , ’r’) as file:

24 key = list(file.keys())[0]

25 elems = file[key][ inds_to_sample]

26 return elems

27

28 # load data at the randomly generated indices

29 masks = load_data_from_h5(os.path.join(h5_save_path , mask_path), inds_to_sample)

30 imgs = load_data_from_h5(os.path.join(h5_save_path , img_path), inds_to_sample)

31 bboxes = load_data_from_h5(os.path.join(h5_save_path , bbox_path), inds_to_sample)

32 binarys = load_data_from_h5(os.path.join(h5_save_path , bin_path), inds_to_sample)

Listing 1: Simple loading

Note: if you fail to implement a suitable loading scheme, you can still proceed with the remainder

of this project by loading all or part of the data directly into memory using the code snippet above,

however, you will loose marks for this working point for doing so.

2.2.3 Implementing a MTL framework

Your MTL framework must follow one of the two schemes:

2

1. Simultaneous learning: The auxiliary task(s) and target task are learnt simultaneously, where

your network ‘shares parameters’ in some layers whereas some layers are specific to the task and

non-shared (usually output layers). Your model may then be considered to have multiple output

branches, each learning a separate task. Once trained, you will only be required to evaluate your

model for the target task. You have freedom to implement any network that you deem to be

suitable, however, please note that you will not be marked for the complexity of your network, but

simply for the correctness of the parameter sharing and multi-output behaviour.

2. Sequential/ transfer learning: The auxiliary task is to be learnt prior to learning the target task.

Parameters are again shared between the two learning stages (e.g. you can learn the auxiliary

task, then freeze parameters in all layers except the last after which you can edit the output layer

according to your target task and train). After training, you will only need to evaluate your model

for the target task. You have freedom to implement any network that you deem to be suitable,

however, please note that you will not be marked for the complexity of your network, but simply

for the correctness of the parameter fixing and layer editing behaviour.

Other things to consider include, but are not limited to, network architecture, training strategies,

regularisers, hyperparamters for both types of tasks. Describe and justify these details in the report, if

considered relevant.

2.2.4 Implementing a baseline network and an ablated version of MTL

The baseline network must be a network capable of performing only your target task. Discuss what

should be considered a fair comparison, in aspects such as network architecture and/or training strategies,

compared with the MTL network.

For the ablation study, you must perform at least one ablation study by adjusting a hyperparameter

of the MTL network. Discuss why this is an interesting ablation study to do. Examples of hyperparam-

eters that can be ablated include 1) weighting(s) of auxiliary task(s); and 2) parameters controlling the

“quantity” or “speed” of the transfer.

2.2.5 Experimenting for network comparison

Design experiments which 1) compare the baseline to the MTL, and 2) compare the ablated version to

the MTL. Things to consider may include, but not limited to, planning data sets and model development,

metrics to use, how to quantitatively and statistically compare these metrics. Describe and justify these

options in the report, if relevant.

2.3 An “open-ended question”

To be awarded the remaining 20% in this project, you need to come up with a new research/study question

to answer, an open-ended question (OEQ). Novelty is encouraged in this part. The submission should

clearly identify and generate a hypothesis (i.e. the research/study question), design experiments that

produce results to answer this question and analyse the obtained experimental results for quantitative

conclusion. This part needs to be built on the MRP and relevant to MTL. Describe the question,

experiment and results clearly and cohesively with the MRP in the report.

Some example research/study questions are given as follows.

What other auxiliary tasks can be used

What exactly in the auxiliary task is used to help the target task

Is there any other things that can help the target task

What can be used to control the “transfer” between the target and auxiliary tasks

......

3 Submission and Marking Criteria

Each submission should include 1) one report in a PDF file and 2) one huddle of code in a single zip file

(50 MB file size limit also applies).

3