SCC366程序代写、MATLAB语言编程代写

日期：2021-11-07 05:34

SCC366 2021/2022

HOLISTIC IMAGE MATCHING

Objectives: To learn holistic image matching based on the example application of face image retrieval:

1) Loading multiple imagesin MATLAB.

2) Representing images as vectors in MATLAB.

3) Implementing the sum ofsquared difference (SSD).

4) Using SSD in image retrieval applications.

Note: The accompanying `ImageRetrievalCode.zip’ file contains example codes and datasets: The input

query image (`QueryImage.png’) is stored in the `QueryImage’ directory while the database images

(`Img001.png’ to `Img200.png’) are stored in `DBImages’ directory.

Our goal is to retrieve all database images (in `DBImages’ directory) which match the query image

(`QueryImage.png’). A given pair of images,say `A’ and `B’, are declared to be a match if their distance is

smaller than a predetermined threshold ?. We will use a fixed number 22,000,000 for the threshold ?: Our

goal can be stated as finding all database images whose distance to the query image is smaller than

?=22000000.

We will adopt the sum of squared difference (SSD) as our distance measure. Since this function is defined

for a pair of vectors, each image (of size HXW) is converted to a vector (of size nX1; n=HW). In our dataset, all

images are 112X92-sized and therefore, n=10,304.

Loading multiple imagesin MATLAB:

Our first step is to load multiple images and convert them into vectors. To facilitate the subsequent

multiple computation of SSD, we will store all database images in a single matrix `DBVecImages’ where

each column vector corresponds to an image.

The MATLAB script `ConvertQueryImage.m’ in `QueryImage’ directory shows how the query image can be

converted to a vector:

2

ConvertQueryImage.m

ImageHeight = 112;

ImageWidth = 92;

%%% Image height and width are fixed at 112 and 92.

VectorSize = ImageHeight*ImageWidth;

%%% `VectorSize’ is the size of vectorised image (VectorSize 1).

QueryImageFilename = 'QueryImage.png';

%%% MATLAB variable `QueryImageFilename’ stores the filename of the query image.

QImg = imread(QueryImageFilename);

%%% loads the image 'QueryImage.png' and stores it as a two-dimensional matrix QImg.

%%% Note: the matrix `QImg’ is in unsigned integer data format (uint8)

%%% which is not suitable for numerical operations, e.g., calculating SSD.

imshow(QImg);

%%% displays the image loaded.

QImg = double(QImg);

%%% converts the uint8 matrix QImg to a double matrix for

%%% subsequent numerical operations.

imagesc(QImg);

%%% displays an image in double format

%%% Note: `imshow’ can show images in uint8 format.

%%% To display an image in double format, we use imagesc function. axis image;

axis off;

QueryVecImage = QImg(:);

%%% `B=A(:)’ converts matrix A to a vector B by concatenating all the columns A.

%%% See `ColonOperatorExample.m’ as discussed shortly.

save QueryVecImage.mat QueryVecImage;

%%% saves the vectorised query image in a MATLAB data file QueryVecImage.mat.

Applying this technique, we will store all the database images in a single matrix. Since there are 200 images in

the `DBImages’ directory, we will use MATLAB’s `dir’ instruction to list all the images in this directory and

convert each image in the `for’ loop. An example usage of `dir’ instruction will be shown shortly

(`ConvertDBImages.m’script).

To store the database images, we will construct a matrix `DBVecImages’ of size 10,304 200: Each column of

`DBVecImages’ will contain a vectorised database image. In MATLAB, a specific column or row of a matrix

can be accessed by using `:’ operator: `A(i,:)’ and `A(:,j)’ denote the i-th row and the j-th column of matrix

A, respectively. Since a column (vector) of a matrix is a vector, `A(:,j)’ is regarded as a single vector while

the transpose of `A(i,:)’ (which is a row vector) is also a vector.

Note: as shown in `ConvertQueryImage.m’ example above, `A(:)’ converts the entire matrix A into a single

vector.

3

` ColonOperatorExample.m’ code shows examples of using `:’ in a matrix. ColonOperatorExample.m

A = rand(3,4);

%%% constructs a matrix (A) of size 3 4 initialized with random numbers.

b = A(:,4);

%%% constructs a vector (b) which is the same to the 4-th column of A.

c = A(2,:);

%%% constructs a row vector (c) which is the same to the 2-nd row of A.

%%% A row vector is not a vector by itself, but its transpose is a vector.

%%% Note, in MATLAB the transpose of a matrix A is denoted as A’.

d = [1,2,3]';

%%% constructs a vector (d) initialized with values 1, 2, and 3.

A(:,2) = d;

%%% replaces the second column of matrix A with the elements of d.

%%% This is how we save a vector as a column of a matrix.

e = [5,5,5,5];

%%% constructs a row vector (e) initialized with 5.

A(3,:) = e;

%%% replaces the third row of matrix A with the elements of e.

f = A(:)

%%% constructs a vector (f) by concatenating all the columns of A.

B = A([1,2],:)

%%% constructs a 2 4 matrix (B) using the first and second rows of A.

C = A([2,3],[3,4])

%%% constructs a 2 2 matrix (C) as using the elements of matrix A

%%% which are contained in rows 2-3 and columns 3-4.

4

Now, we are ready to convert all database images to vectors and store them as column vectors of a matrix:

`ConvertDBImages.m’ in `DBImages’ directory.

ConvertDBImages.m

ImageHeight = 112;

ImageWidth = 92;

VectorSize = ImageHeight*ImageWidth;

ImageFiles = dir('*.png');

%%% extracts information of all files whose filename end with `.png’.

NumDBImages = length(ImageFiles);

%%% `length’ returns the number of elements of the data structure `ImageFiles’.

%%% In this example, `length(ImageFiles)’ is the number of images (200).

DBVecImages = zeros(VectorSize,NumDBImages);

%%% This is the matrix that will store all vectorised images:

%%% Each column of DBVecImages will store a vectorised image.

%%% `VectorSize’ is 10304 (92 112).

for i=1:NumDBImages

DBImg = imread(ImageFiles(i).name);

%%% `ImageFiles(i).name’ contains the filename of the i-th image.

DBImg = double(DBImg);

%%% `DBImg = double(DBImg)’ converts the uint8 data matrix DBImg to double format.

end

%%% to be written by you from here.

…

%%% to be written by you till here.

save DBVecImages.mat DBVecImages;

%%% saves DBVecImages in a MATLAB data file DBVecImages.mat.

Exercise 1

Complete the MATLAB script `ConvertDBImages.m’ by writing the missing lines (marked with red dots

above) and run it: It will generate `DBVecImages.mat’ containing a MATLAB matrix variable

`DBVecImages’. Each column of `DBVecImages’ correspondsto a vectorised database image.

You can verify the successful completion of this step by running `DisplayVecImages.m’ script in

`DBImages’ directory. Ifsuccessful, thisscript will display an image given below.

5

6

Even if you do not complete this exercise, you can proceed to the next exercises. In this case, exercise

3 requires the pre-calculated data files `QueryVecImage.mat’ and `DBVecImages.mat’ provided in

`VectorizedImages’ directory.

Image matching using SSD:

Our next step is to build a function SSD. For a given pair of vectors a and b, the SSD is defined as

Here, the two input vectors are of size n 1. In our face image retrieval example n is fixed at 10,304.

Exercise 2

Write a MATLAB function (‘SSD.m‘) that receives two vectors and returns the SSD of these vectors.

You can verify the successful completion of this step by running `TestSSD.m’ script. If successful, this script

will print out:

Even if you do not complete this exercise, you can proceed to the next exercises. In this case, exercise 3

requires a pre-compiled MATLAB script `SSD.mexw64’ provided in `SSDCode’ directory.

Given `SSD.m’ script (or `SSD.mexw64’ code) and MATLAB data files containing vectorised query

(`QueryVecImage.mat’) and database images (`DBVecImages.mat’), we can immediately calculate the SSDs

of the query and the database images.

Note: the query image is given as a vector while the database images are stored as columns of the matrix

`DBVecImages’. The SSD between the vectorised query image and the i-th image can be calculated by

accessing the i-th column of the `DBVecImages’ as a vector. See

`ColonOperatorExample.m’ for examples of accessing (extracting) column (or row) vectors in a matrix.

SSD=2.862500

SSD=8.806100

7

Exercise 3

Complete the MATLAB script `CalculateQuerySSD.m’ by writing the missing lines (marked with red dots below)

and run it: It will generate `QuerySSD.mat’ containing a MATLAB vector variable

`SSDVal’ which contains the distances between the input query image and the database images.

CalculateQuerySSD.m

ImageHeight = 112;

ImageWidth = 92;

load QueryVecImage.mat QueryVecImage;

%%% loads vectorised query image.

load DBVecImages.mat DBVecImages;

%%% load database images stored in DBVecImages.

[~,NumDBImages] = size(DBVecImages);

%%% For the matrix A, `[MatR,MatC] = size(A)’ returns the numbers of rows and columns,

%%% to MatR and MatC, respectively.

SSDVal = zeros(NumDBImages,1);

%%% allocates memory for a vector variable SSDVal and initializes it with zeros.

%%% to be written by you from here.

…

%%% to be written by you till here.

save QuerySSD.mat SSDVal;

%%% saves SSDVal vector that contains the calculated SSD values.

for i=1:10

display(sprintf('Image%03d: SSD value = %f',i,SSDVal(i)));

end

%%% displays the SSD values of the first 10 database images.

You can verify the successful completion of this step. If successful, this script will print out:

Image001: SSD value = 25168263.000000

Image002: SSD value = 24790550.000000

Image003: SSD value = 29749814.000000

Image004: SSD value = 26188179.000000

Image005: SSD value = 39443834.000000

Image006: SSD value = 24414565.000000

Image007: SSD value = 21459433.000000

Image008: SSD value = 22668506.000000

Image009: SSD value = 22176936.000000

Image010: SSD value = 19669056.000000

8

Exercise 4

Our final exercise is to retrieve all images in the database which have distance (from the query image) smaller

than our threshold ?=22,000,000 and display them.

Complete the MATLAB script `ImageRetrieval.m’ by writing the missing lines (marked with red dots below)

and run it: It will display the retrieved images.

If successful, this script will display an image given below:

Note: the retrieved images are not sorted: the left-most image is not the closest. One can re-arrange the

images in increasing SSD from the query image.

Conclusion

In this assignment, the examples and exercises were developed to help you to understand how to setup and

perform holistic image matching-based face image retrieval using the sum ofsquared difference (SSD).

ImageRetrieval.m

ImageHeight = 112;

ImageWidth = 92;

Tau = 22000000;

load QueryVecImage.mat QueryVecImage;

load DBVecImages.mat DBVecImages;

load QuerySSD.mat SSDVal;

%%% to be written by you from here.

…

%%% to be written by you till here.

figure(1);

DisplayPatches(RetrievedVecImages', NumRetrievedImages, ImageHeight, ImageWidth,

NumRetrievedImages);

%%% `NumRetrievedImages’ is the number of retrieved images which are

%%% the database images whose distance from the query image is smaller than Tau.

%%% `RetrievedVecImages’ is a matrix of size 10,304?`NumRetrievedImages’:

%%% Each column in this matrix contains a retrieved image.