GRPC留学生代做、C++编程语言调试、代写c/c++、代写system

日期：2019-07-19 08:27

GRPC and Distributed Systems

Forward

In this project, you will design and implement a simple distributed file system (DFS). First, you will develop several file transfer

protocols using gRPC and Protocol Buffers. Next, you will incorporate a weakly consistent synchronization system to manage

cache consistency between multiple clients and a single server. The system should be able to handle both binary and textbased

files.

Your source code will use a combination of C++14, gRPC, and Protocol Buffers to complete the implementation.

Setup

You can clone the code in the Project 4 repository with the command:

git clone https://github.gatech.edu/gios-sum-19/pr4.git

Submission Instructions

Submit all code through the submit.py script given at the top level of the repository. For instructions on how to submit

individual components of the assignment, see the instructions below.

For this assignment, you may submit your code up to 10 times in 24 hours. After the deadline, we download your last

submission before the deadline, review your submission, and assign a grade. There is no limit to the number of times you

may submit your readme-student.md file.

After submitting, you may double-check the results of your submission by visiting the Udacity/GT autograding website and

going to the student portal.

Readme

Throughout the project, we encourage you to keep notes on what you have done, how you have approached each part of

the project, and what resources you used in preparing your work. We have provided you with a prototype file, readmestudent.md

that you should use throughout the project.

You may submit your readme-student.md file with the command:

python submit.py readme

At the prompt, please provide your GT username and password.

Raw Blame History

7/15/2019 pr4/readme.md at master · gios-sum-19/pr4

https://github.gatech.edu/gios-sum-19/pr4/blob/master/readme.md 2/7

If this is your first time submitting, the program will then ask you if you want to save the JWT. If you reply yes, it will save a

token on your filesystem so that you don't have to provide your username and password each time that you submit.

The Udacity site will store your readme-student.md file in a database, where it will be used during grading. The submit script

will acknowledge receipt of your README file. For this project, like in Project 3, you only need to submit one README file for

both parts of the project.

Note: you may choose to submit a PDF version of this file (readme-student.pdf) in place of the markdown version. The

submission script will automatically detect and submit this file if it is present in the project root directory. If you submit both

files, we will give the PDF version preference.

Directions

Part 1 - Building the RPC protocol service

In Part 1, you will build a series of remote procedure calls (RPC) and message types that will fetch, store, list, and get

attributes for files on a remote server. The SunRPC implementation (ONC RPC) and XDR interface definition language (IDL)

you learned about in the lectures are currently deprecated in favor of more modern implementations of the concepts behind

RPC and IDL. There are many RPC replacements, including TI-RPC, Finagle, Thrift, and Cap'n Proto. However, in this

assignment, we will use gRPC for RPC services and Protocol Buffers as the definition language. The core gRPC library is

written in C but supports multiple languages, including C++, Java, Go, and others. It is actively developed and in use at

several organizations, such as Google, Square, Netflix, Juniper, Cisco, and Dropbox. In this assignment, we will use the gRPC

C++ API.

Part 1 Goals

The goal of part 1 is to generate an RPC service that will perform the following operations on a file:

Fetch a file from a remote server and transfer its contents via gRPC

Store a file to a remote server and transfer its data via gRPC

List all files on the remote server:

For this assignment, the server is only required to contain files in a single directory; it is not necessary to manage

nested directories.

The file listing should include the file name and the modified time (mtime) of the file data in seconds from the

epoch.

Get the following attributes for a file on the remote server:

Size

Modified Time

Creation Time

The client should be able to request each of the operations described above for binary and text-based files. The server will

respond to those requests using the gRPC service methods you specify in your proto buffer definition file.

Finally, the client should recognize when the server has timed out. gRPC can signal the server using a deadline timeout

as described in this gRPC article. You should ensure that your client recognizes this timeout signal.

Part 1 Sequence Diagram

A sequence diagram of the expected interactions in part 1 is available in the docs/part1-sequence.pdf file of this repository.

Protocol Buffers and gRPC

7/15/2019 pr4/readme.md at master · gios-sum-19/pr4

https://github.gatech.edu/gios-sum-19/pr4/blob/master/readme.md 3/7

To begin part 1, you should first familiarize yourself with the basics of using Protocol Buffers. In particular, you should focus

on the use of RPC service definitions and message type definitions for the request and response types that are used by the

RPC services.

You will then create your protocol in the dfs-service.proto file inside the project repository. There are several required

services and message types described in the proto file that you should implement, but you may add as many additional

methods and/or message types that you deem necessary. What you name those services and message types is also up to

your discretion.

To autogenerate the gRPC and Protocol Buffer class and header files, we’ve provided a Makefile command that will take care

of that for you. When you are ready to generate your protobuf/gRPC files, run the following command from the root of the

repository:

make protos

You will find the results of that command in the part1/proto-src directory of the repository. You should familiarize yourself

with the results in that directory, but you won’t need to, and should not, make any changes to those files. Your job will be to

override the service methods in your dfslib-servernode-p1.cpp source file.

Once you have familiarized yourself with Protocol Buffers, you should next familiarize yourself with the C++ API for gRPC. In

particular, pay close attention to how the server implementation overrides methods, and the client makes calls to the RPC

service for streaming message types.

You do not need to concern yourself with asynchronous gRPC, we will only be working with the synchronous version in

this project.

Part 1 Structure

All of the part 1 files are available in the part1 directory. You will find several source files in that directory, but you are only

responsible for adjusting and submitting the difslib-* files inside part1 . The rest of the source files provide the

supporting structure for the program. You may change any of the other source files for your testing purposes, but they will

not be submitted as a part of your grade.

In each of the files to be modified, you will find additional instructions and hints on how you should approach the contents

of that file. The following comment marker precedes each tip in the source code:

// STUDENT INSTRUCTION:

Source code file descriptions:

src/dfs-client-p1.[cpp,h] - the CLI executable for the client side.

src/dfs-server-p1.[cpp,h] - the CLI executable for the server side.

src/dfs-clientnode.[cpp,h] - the parent class for the client node library file that you will override. All of the methods

you will override are documented in the dfslib-clientnode-p1.h file you will modify.

src/dfs-utils.h - A header file of utilities used by the executables. You may change this, but note that this file is not

submitted to Bonnie. There is a separate dfs-shared file you may use for your utilities.

dfs-service.proto - TO BE MODIFIED Add your proto buffer service and message types to this file, then run the make

protos command to generate the source.

dfslib-servernode-p1.[cpp,h] - TO BE MODIFIED - Override your gRPC service methods in this file by adding them to

the DFSServerImpl class. The service method signatures can be found in the proto-src/dfs-service.grpc.pb.h file

generated by the make protos command you ran earlier.

7/15/2019 pr4/readme.md at master · gios-sum-19/pr4

https://github.gatech.edu/gios-sum-19/pr4/blob/master/readme.md 4/7

dfslib-clientnode-p1.[cpp,h] - TO BE MODIFIED - Add your client-side calls to the gRPC service in this file. We’ve

provided the basic structure and method calls expected by the client executable. However, you may add any additional

declarations and definitions that you deem necessary.

dfslib-shared-p1.[cpp,h] - TO BE MODIFIED - Add any shared code or utilities that you need in this file. The shared

header is available on both the client and server side.

Part 1 Compiling and Running

To compile the source code in Part 1, you may use the Makefile in the root of the repository and run:

make part1

Or, you may change to the part1 directory and run make .

For a list of all make commands available, run make in the root of the repository.

To run the executables, see the usage instructions in their respective files.

In most cases, you'll start the server with:

./bin/dfs-server-p1

The client is then used to fetch, store, list, and stat files. For example:

./bin/dfs-client-p1 fetch gt-campanile.jpg

Part 1 Submitting

To submit part 1, run the following from the root of the repository:

python submit.py part1

Part 2 - Completing the DFS

Now that you have a working gRPC service, we will turn our focus towards completing a rudimentary DFS. For this

assignment, we’ll apply a weakly consistent cache strategy to the RPC calls you already created in Part 1. This is similar to the

approach used by the Andrew File System (AFS). To keep things simple, we’ll focus on whole-file caching for the client-side

and a simple lock strategy on the server-side to ensure that only one client may write to the server at any given time.

Part 2 Goals

For this assignment, your consistency model should adhere to the following expectations:

Whole-file caching. The client should cache whole files on the client-side (i.e., do not concern yourself with partial file

caches). Read and write operations should be applied to local files and only update the server when a file has been

modified or created on the client.

One Creator/Writer per File (i.e., writer locks). Clients should request a file write lock from the server before pushing. If

they are not able to obtain a lock, the attempt should fail. The server should keep track of which client holds a lock to a

particular file, then release that lock after the client has successfully stored a file to the server.

CRC Checksums. To determine if a file has changed between the server and the client, you may use a CRC checksum

function we have provided for you. Any differences in the checksums between the server and the client constitute a

change. The CRC function requires a file path and a table that we’ve already set up for you. The return value will be a

uint32_t value that you can use in your gRPC message types. An example of how to obtain the checksum follows:

7/15/2019 pr4/readme.md at master · gios-sum-19/pr4

https://github.gatech.edu/gios-sum-19/pr4/blob/master/readme.md 5/7

std::uint32_t crc = file_checksum(filepath, this->crc_table);

Note that you can copy the code from part1 for your Store, Fetch, List, and Stat methods to part2. However, please note

that you will most likely need to adjust those methods to meet the requirements of the DFS implementation.

Date based Sequences. If a file has changed between the server and the client, then the last modified timestamp should

win. In other words, if the server has a more recent timestamp, then the client should fetch it from the server. If the client

has a more recent timestamp, then the client should store it on the server. If there is no change in the modified time for

a file, the client should do nothing. Keep in mind, when storing data, you must first request a write lock as described

earlier.

Part 2 Structure

The file structure for Part 2 is identical to Part 1. As with Part 1, you are only responsible for adjusting the dfslib-* files in

the part2 directory.

Two threads on the client side will run concurrently. You will need to synchronize these threads and their access to the server.

The watcher thread uses the inotify system commands to monitor client directories for you. We’ve already provided the

structural components to manage the events. However, you will need to make the appropriate changes to ensure that file

notification events coordinate with the synchronization timer described next. An event callback function is provided in the

dfslib-clientnode-p2.cpp file to assist you with that; read the STUDENT INSTRUCTION comments carefully as they provide

some hints on how to manage that process.

The sync thread uses a simple timer to connect with the server every 3 seconds, similar to how NFS connects to a server. The

client should request a list of files from the server, along with their last modified time (mtime), then synchronize the files

between the client and the server. The client should fetch, store, or do nothing based on the goals discussed earlier.

Part 2 Sequence Diagram

A sequence diagram of the expected interactions for part 2 is available in the docs/part2-sequence.pdf file of this repository.

Part 2 Compiling

To compile the source code in Part 2, you may use the Makefile in the root of the repository and run:

make part2

Or, you may change to the part2 directory and run make .

For a list of all make commands available, run make in the root of the repository.

To run the executables, see the usage instructions in their respective files.

In most cases, you'll start the server with something similar to the following:

./bin/dfs-server-p2

The client should then mount to the client path. For example:

./bin/dfs-client-p2 mount

The above command will mount the client to the mnt/client directory, then start the watcher and sync timer threads.

Changes to the mount path should then synchronize to the server and any other clients that have mounted.

As in Part 1, the client should also continue to accept individual commands, such as fetch, store, list, and stat.

Part 2 Submitting

7/15/2019 pr4/readme.md at master · gios-sum-19/pr4

https://github.gatech.edu/gios-sum-19/pr4/blob/master/readme.md 6/7

To submit part 2, run the following from the root of the repository:

python submit.py part2

References

Relevant lecture material

P4L1 Remote Procedure Calls

gRPC and Protocol Buffer resources

gRPC C++ Reference

Protocol Buffers 3 Language Guide

gRPC C++ Examples

gRPC C++ Tutorial

Protobuffers Scalar types

gRPC Status Codes

gRPC Deadline

Rubric

Your project will be graded at least on the following items:

Interface specification (.proto)

Service implementation

gRPC initiation and binding

Proper handling of deadline timeouts

Proper clean up of memory and gRPC resources

Proper communication with the server

Proper request and management of write locks

Proper synchronization of files between multiple clients and a single server

Insightful observations in the Readme file and suggestions for improving the class for future semesters

gRPC Implementation (40 points)

Full credit requires: code compiles successfully, does not crash, files fully transmitted, basic safety checks, and proper use of

gRPC - including the ability to get, store, and list files, along with the ability to recognize a timeout. Note that the automated

tests will test some of these automatically, but graders may execute additional tests of these requirements.

DFS Implementation (50 points)

Full credit requires: code compiles successfully, does not crash, files fully transmitted, basic safety checks, proper use of

gRPC, write locks properly handled, cache properly handled, synchronization of sync and inotify threads properly handled,

and synchronization of multiple clients to a single server. Note that the automated tests will test some of these automatically,

but graders may execute additional tests of these requirements.

README (10 points + 5 point extra credit opportunity)

Clearly demonstrates your understanding of what you did and why - we want to see your design and your explanation

of the choices that you made and why you made those choices. (4 points)

A description of the flow of control for your code; we strongly suggest that you use graphics here, but a thorough

textual explanation is sufficient. (2 points)

7/15/2019 pr4/readme.md at master · gios-sum-19/pr4

https://github.gatech.edu/gios-sum-19/pr4/blob/master/readme.md 7/7

A brief explanation of how you implemented and tested your code. (2 points)

References any external materials that you consulted during your development process (2 points)

Suggestions on how you would improve the documentation, sample code, testing, or other aspects of the project (up to

5 points extra credit available for noteworthy suggestions here, e.g., actual descriptions of how you would change

things, sample code, code for tests, etc.) We do not give extra credit for simply reporting an issue - we're looking for

actionable suggestions on how to improve things.

Questions

For all questions, please use the class Piazza forum or the class Slack channel so that TA's and other students can assist you.