MSE 5760代做、代写C/C++，Java程序

日期：2025-05-05 08:13

MSE 5760: Spring 2025 HW 6 (due 05/04/25)

Topic: Autoencoders (AE) and Variational Autoencoders (VAE)

Background:

In this final homework, you will build a deep autoencoder, a convolutional

autoencoder and a denoising autoencoder to reconstruct images of an isotropic composite

with different volume fractions of fibers distributed in the matrix. Five different volume

fraction of fibers are represented in the dataset and these form five different class labels for

the composites. After the initial practice with AEs and reconstruction of images using latent

vectors, you will build a VAE to examine the same dataset. After training the VAE (as best

as you can using the free colab resources to reproduces images), you will use it to generate

new images by randomly sampling datapoints from the learned probability distribution of

the data in latent space. Finally, you will build a conditional VAE to not only generate new

images but generate them for arbitrary volume fractions of fibers in the composite.

The entire dataset containing 10,000 images of composites with five classes of

volume fractions of fibers was built by Zequn He (currently a Ph.D. student in MEAM in

Prof. Celia Reina’s group who helped put together this course in Summer 2022 by designing

all the labs and homework sets). Each image in the dataset shows three fibers of different

volumes with circular cross sections. Periodic boundary conditions were used to generate

the images. Hence, in some images, the three fiber particles may appear broken up into

more than three pieces. The total cross sectional area of all the fibers in each image can,

however, be divided equally among three fibers. Please do not use this dataset for other

work or share it on data portals without prior permission from Zequn He

(hezequn@seas.upenn.edu).

Due to the large demands on memory and the intricacies of the AE-VAE

architecture, the results obtained will not be of the same level of accuracy and quality that

was possible in the previous homework sets. No train/test split is recommended as all

10,000 images are used for training purposes. You may, however, carry out further analysis

using train/test split or by tuning the hyperparameters or changing the architecture for

bonus points. The maximum bonus points awarded for this homework will be 5.

**********************************Please Note****************************

Sample codes for building the AE, VAE and a conditional GAN were provided in

Lab 6. There is no separate notebook provided for the homework and students will

have to prepare one. Tensorflow and keras were used in Lab 6 and is recommended

for this homework. You are welcome to use other libraries such as pytorch.

************************************************************************

1. Model 1: Deep Autoencoder model (20 points)

Import the needed libraries. Load the original dataset from canvas. Check the

dimensions of each loaded image for consistency. Scale the images.

1.1 Print the class labels and the number of images in each class. Print the shape of

the input tensor representing images and the shape of the vector representing the

class labels. (2 points)

1.1. A measure equivalent to the volume fraction of fibers in each composite image is

the mean pixel value of the image. As the images are of low-resolution, you may

notice a slight discrepancy in the assigned class value of the image and the

calculated mean pixel intensity. As the resolution of images increases, there will be

negligible difference between the assigned class label and the pixel mean of the

image. Henceforth, we shall use the pixel mean (PM) intensity of the images to be

the class label. Print a representative sample of ten images showing the volume

fraction of fibers in the composite along with the PM value of the image. (3 points)

1.2. Build the following deep AE using the latent dimension value = 64.

(a) Let the first layer of the encoder have 256 neurons.

(b) Let the second layer of the encoder have 128 neurons.

(c) Let the last layer of the encoder be the context or latent vector.

(d) Use ReLU for the activation function in all of the above layers.

(e) Build a deep decoder with its input being the context layer of the encoder.

(f) Build two more layers of the decoder with 128 and 256 neurons, respectively.

These two layers can use the ReLU activation function.

(g) Build the final layer of the decoder such that its output is compatible with the

reconstruction of the original input shape tensor. Use sigmoid activation for the

final output layer of the decoder.

(h) Use ADAM as your optimizer and a standard learning rate. Let the loss be the

mean square error loss. Compile the AE and train it for at least 50 epochs.

(10 points)

1.3. Print the summary of the encoder and decoder blocks showing the output shape of

each layer along with the number of parameters that need to be trained. Monitor

and print the lossfor each epoch. Plot the loss as a function of the epochs. (2 points)

1.4. Plot the first ten reconstructed images showing both the original and reconstructed

images. (3 points)

2. Model 2: Convolutional Autoencoder model (20 points)

2.1 Build the following convolutional AE with the latent dimension = 64

(a) In the first convolution block of the encoder, use 8 filters with 3x3 kernels,

ReLU activation and zero padding. Apply max pooling layer with a kernel of

size 2.

(b) In the second convolution block use 16 filters with 3x3 kernels, ReLU activation

and zero padding. Apply max pooling layer with a kernel of size 2.

(c) In the third layer of the encoder use 32 filters with 3x3 kernels, ReLU activation

and zero padding. Apply max pooling layer with a kernel of size 2.

(d) Flatten the obtained feature map and then use a Dense layer with ReLU

activation function to extract the latent variables.

(d) Build the decoder in the reverse order of the encoder filters with the latent

output layer of the encoder serving as the input to the decoder part.

(e) Use ADAM as your optimizer and a standard learning rate. Let the loss be the

mean square error loss. Compile the convolutional AE and train it for at least

50 epochs.

(10 points)

2.2 Print the summary of the encoder and decoder blocks showing the output shape of

each layer along with the number of parameters that need to be trained. Monitor

and print the lossfor each epoch. Plot the loss as a function of the epochs. (5 points)

2.3 Plot the first ten reconstructed images showing both the original and reconstructed

images. (5 points)

3. Model 3: Denoising convolutional Autoencoder model (15 points)

3.1 Add a Gaussian noise to each image. Choose a Gaussian with a mean of zero and a

small standard deviation, typically ~ 0.2. Plot a sample of five original images with

noise. (3 points)

3.2 Use the same convolutional autoencoder as in Problem 2 but with noisy images fed

to the encoder. Train and display all the information as in 2.2 and 2.3.

(12 points)

4. Model 4: Variational Autoencoder model (25 points)

4.1 Set the latent dimension of the VAE be 64. Build a convolutional autoencoder with

the following architecture. Set the first block to have 32 filters, 3x3 kernels with

stride = 2 and zero padding.

4.2 Build the second block with 64 filters, 3x3 kernels, stride =2 and zero padding. Use

ReLU in both blocks. Apply max pooling layer with kernel of size 2x2.

4.3 Build an appropriate output layer of the encoder that captures the latent space

probability distribution.

4.4 Define the reparametrized mean and variance of this distribution.

4.5 Build the convolutional decoder in reverse order. Apply the same kernels, stride

and padding as in the encoder above. Choose the output layer of the decoder and

apply the appropriate activation function.

4.6 Compile and train the model. Monitor the reconstruction loss, Kullback-Liebler

loss and the total loss. Plot all three quantities for 500 epochs. (10 points)

4.7 Plot the first ten reconstructed images along with their originals. (5 points)

4.8 Generate ten random latent variables from a standard Gaussian with mean zero and

unit variance. Display the generated images from these random values of the latent

vector. Comment on the quality of your results and how it may differ from the input

images. Mention at least one improvement that can be implemented which may

improve the results. (3+3+4=10 points)

5. Model 5: Conditional Variational Autoencoder model (20 points)

A conditional VAE differs from a VAE by allowing for an extra input

variable to both the encoder and the decoder as shown below. The extra label could

be a class label, ‘c’ for each image. This extra label will enable one to infer the

conditional probability that describes the latent vector conditioned on the class label

‘c’ of the input. In VAE, using the variational inference principle, one infers the

Gaussian distribution (by learning its mean and variance) of the latent vector

representing each input ‘x’. In conditional VAE, one infers the Gaussian

conditional distribution of the latent vector conditioned on the extra input variable

‘label’.

For the dataset used in this homework, there are two advantages of the

conditional VAE compared to the VAE: (i) the conditional VAE provides a cheap

way to validate the model by comparing the pixel mean of the generated images

with the conditional class label values (pixel mean) in latent space used to generate

the images. (ii) The trained conditional VAE can be used to generate images of

composites with arbitrary volume fraction of fibers with sufficient confidence once

the validation is done satisfactorily.

A conditional VAE. (source: https://ijdykeman.github.io/ml/2016/12/21/cvae.html)

A good explanation of the conditional VAE in addition to the resource cited in the

figure above is this: https://agustinus.kristia.de/techblog/2016/12/17/conditional vae/.

A conditional GAN (cGAN) toy problem was shown in Lab 6 where the volume

fraction (replaced by pixel mean for cheaper model validation) was the design

parameter, and thus, the condition input into the cGAN. In this question, you will

build a conditional VAE for the same task of generating new images of composites

as in Problem 4 by randomly choosing points in the latent space. Since each point

in the latent space represents a conditional Gaussian distribution, it also has a class

label. Therefore, it becomes possible to calculate the pixel mean of a generated

image and compare it with the target ‘c’ value of the random point in latent space.

It is recommended that students familiarize themselves with the code for providing

the input to the cGAN with class labels and follow similar logic for building the

conditional VAE. You may also seek help from the TA’s if necessary.

5.1 Create an array that contains both images and labels (the pixel mean of each image).

Note the label here is the condition and it should be stored in the additional channel

of each image.

5.2 Use the same structure, activation functions and optimizer as the one used to build

the VAE in Problem 4. Print the summary of the encoder and decoder blocks

showing the output shape of each layer along with the number of parameters that

need to be trained. (5 points)

5.3 Train the cVAE for 500 epochs. Plot the reconstruction loss, Kullback-Liebler loss

and the total loss. Plot the first ten reconstructed images along with their originals.

Include values of the pixel mean for both sets of images. (5 points)

5.4 Generate 10 fake conditions (i.e., ten volume fractions represented as pixel means

evenly spaced within the range 0.1 to 0.4 as used in Lab 6) for image generation.

Print the shape of the generated latent variable. Print the target volume fraction (or

pixel mean). Show the shape of the array that combines the latent variables and fake

conditions. Print the shape of the generated image tensor. (2 points)

5.5 Plot the 10 generated images. For each image show the generated condition (the

pixel mean of each image generated in 5.4) and the pixel mean calculated from the

image itself. (3 points)

5.6 Compare the set of generated images from the conditional VAE with the ones

obtained in Lab 6 using cGAN. Comment on their differences and analyze the

possible causes for the differences. (5 points)