如果您想了解COMP1047Part-2:ComputerNetworks的知识，那么本篇文章将是您的不二之选。同时我们将深入剖析ARecipeforTrainingNeuralNetworks[中文

如果您想了解COMP1047 Part-2: Computer Networks的知识，那么本篇文章将是您的不二之选。同时我们将深入剖析A Recipe for Training Neural Networks [中文翻译，part 1]、A Self-Organized Computer Virus Demo in C、Chapter 1: A Tour of Computer Systems、COEN 146: Computer Networks的各个方面，并给出实际的案例分析，希望能帮助到您！

• COMP1047 Part-2: Computer Networks
• A Recipe for Training Neural Networks [中文翻译，part 1]
• A Self-Organized Computer Virus Demo in C
• Chapter 1: A Tour of Computer Systems
• COEN 146: Computer Networks

COMP1047 Part-2: Computer Networks

COMP1047
University of Nottingham Ningbo China
SCHOOL OF COMPUTER SCIENCE
A LEVEL 1 MODULE, SPRING SEMESTER 2023 –2024
COMP1047: Systems and Architecture (AE1SYS)
Coursework Part-2: Computer Networks
SUBMISSION DEADLINE: Thursday 2
nd May 2024, 23:59:59 GMT+8
1 Synopsis
This is Par-2 of the COMP1047 coursework. This Part is about network performance
evaluation both simulation and report writing. You should answer the TWO (2) questions
below, before submitting them in Moodle and CS repository. For the NS-2 coding problem,
you are advised to develop and analyze your TRACE files using AWK adopted in the lab
sessions in this module. The total mark of this Part is [30 MARKS].
Note that the nature of this Part of coursework is exploratory to a certain extent, given its
time to complete (more than 2 weeks). Thus, the solutions of the questions cannot be directly
obtained from the lecture or lab materials, but require students to explore further from public
resources, or recall from what have been learnt from prerequisite modules. Nonetheless, the
module content of COMP1047 should be recognized as the essential foundation to achieve the
proper answering to the questions of this Part of coursework.
2 Deliverable
Submit the following THREE (3) uncompressed files to COMP1047 Moodle page, where you
will find a submission portal named “Submission for Part-2: Computer Networks” to be opened
later.
1.COMP1047CWP2-‘YourName’-‘YourID’-Q1.tcl for the code corresponding to Question

1. Example: COMP1047CWP1-JaneDoe-20511234-Q1.tcl
   2.COMP1047CWP2-‘YourName’-‘YourID’-Q2.xlsx, in which you only need to write down
   your project name (AE1SYS-CWP2-YourID) in the csproject. Details can be found in Question
   1’s description.
   3.COMP1047CWP2-‘YourName’-‘YourID’-Q2.pdf, in which you only need to write down
   your project name (AE1SYS-CWP2-YourID) in the csproject. Details can be found in Question
   2’s description.
   Unable to follow the file naming conventions would lead to mark deduction. Late submission rules apply, as indicated in the accompanying coursework issue sheet.
2. Assessment
3. This Part has a total of 30 marks, which constitutes 60% of the total coursework marks,
   equivalent to 30% of the total module marks. Individual marks are shown along each
   question.
4. Evaluation rubrics for Questions is provided in their problem description.
5. For enquiries on doubts or issues encountered when doing your coursework, please use
   the Moodle discussion forum, or directly email Dr. Fazl Ullah
   Question 1 [15 Marks]
   Description
   This project provides insights on analyzing network performances such as delay, throughput, and
   packet drops of your designed network. It presents an experimental methodology to obtain an
   estimate of average delay, throughput, and packet drops for packets of variable payload size. You
   are required to use the TWO transport layer protocols, the TCP and UDP as we studied in Week3 class.
   The Scenario 5 Marks
   Implement a circular wireless network topology of at least 10 nodes. In these nodes 20% will be
   sender nodes and 20% will be receiver nodes and the remaining 60% nodes will be the relay
   nodes. You need to use TCP as transport layer protocol for 10% sender-receiver pair and UDP
   for the ramming 10% sender-receiver pair.
   Note that the senders and receiver will be selected by a random function. You have to use the
   random function only once.
   Simulation 3 Marks
   You need to run your simulation for 1000 seconds to generate TRACE (.tr) and NAM (.nam) files
   of your designed scenario. Also, you are required to run the simulation with the same topology 20
   times and find the use the average values in trace fie for performance analysis.
   Hint: You can use shell script to automate simulation for 20 times.
   Performance Analysis 1 Mark
   You need to analyze your designed network based on delay, throughput, and packet drop using
   AWK or Shell or Perl scripts or scripting language you want.
   Delay
   You find the end-to-end delay in the network [1 Mark] and identified the reason [2 Marks]
   Throughput
   You find the total throughput in the network [1 Mark] and identified the reason of high or low
   throughput [2 Marks]
   Packet Drop
   You find the packet drop ratio in the network [1 Mark] and you identified the reason [2 Marks]
   Excel File [2 Marks]
   Once you run and obtain the average simulation, then you need to analyze the Delay, Throughput,
   and Packet Drop. You need to save the values of Delay, Throughput, and Packet Drop in the Excel
   file.
   Note: You can use any application for graph generation and data presentation.
   Question 2 [10 Marks]
   The Scenario
   After performing the simulations and analyzing the designed networks, you are required
   to write a report using the attached template.
   Your Task
   Title [10-20 Words]
   A nice title that reflect what you have written in this report.
   Summary [150-250 words]
   A short summary that give an idea of your work
   Introduction [~600 words]
   A brief introduction about the simulated network. You may giver references from the
   literature from similar networks.
   Proposed Work [400 words]
   Write about the designed network and the protocols used in it. The details about the
   simulation scenario and the working procedures.
   Results and Discussion [500 words]
   You obtained the results from simulation and need to present in the form of graphs. Explain
   the graphs and
   Conclusion [100-150 words]
   Write a short conclusion, what you did and how can you enhance the current work in the
   near future.
   References (if any)
   [1]. Reference 1
   [2]. Reference [2], etc.
   Also, it is important to submit all your files on time to our CS git server, namely
   csprojects.nottingham.edu.cn. To upload to csprojects, please:
   a. Create a new project named AE1SYS-CWP2-YourID, e.g., AE1SYS-CWP2-20511234.
   b. Upload your files into the project, and name them as TCL-YourID, xlx-YourID and
   PDF-YourID, respectively.
   c. You must add Dr. Fazl Ullah as the project Maintainer in order for your project to be
   accessed. This can be done by Project Information → Members → Invite
   member, then input his email address (fazl.ullah@nottingham.edu.cn), and set him as
   the Maintainer in the ‘Select a role’ menu.
   Evaluation
   Basic marking criteria are correctness, abundance, and depth. For detailed
   rubrics, an example indicator is provided in the table below, to give you n direct impression on
   how much effort should be corresponding to which level of grades.
   Level Example indicator
   Fail Nothing done.
   Poor Provided the names only.
   OK Provided the names, with one or two sentences of description.
   Good In addition to OK level, provided extended elaboration trying to
   help the evaluator understand what is talked about.
   Excellent The information conveyed is correct and abundant. For question 2,
   in-depth discussion and/or analysis is provided. High quality usage
   of citations.
   Besides the above, each item below will incur a penalty of -1 mark:
   a. Not adding Dr. Fazl Ullah as the maintainer to access your project.
   b. Wrong project name, or file names uploaded, especially
   WX：codinghelp

A Recipe for Training Neural Networks [中文翻译，part 1]

最近拜读大神 Karpathy 的经验之谈 A Recipe for Training Neural Networks  https://karpathy.github.io/2019/04/25/recipe/，这个秘籍对很多深度学习算法训练过程中遇到的各自问题进行了总结，并提出了很多很好的建议，翻译于此，希望能够帮助更多人学到这些内容。

译文如下：

几周前，我发布了一条关于 “最常见的神经网络错误” 的推文，列出了一些与训练神经网络相关的常见问题。这条推文得到了比我预期的要多得多的认同（包括网络研讨会：)）。显然，很多人个人遇到了 “卷积层是这样工作的” 和 “我们的卷积网络达到最先进结果” 之间的巨大差距。

所以我认为如果清空我尘土飞扬的博客并将这个推文扩展到这个主题应该得到的长篇形式应该是有趣的事情。然而，与其列举常见的错误或深入分析它们，我更想深入挖掘并讨论如何避免出现这些错误（或者非常快速地修复它们）。这样做的关键是遵循某个过程，据我所知，这个过程并没有文档记录下来。让我们从促使我做这个的两个重要发现开始吧。

1. 神经网络训练的困难是一种假象

据称很容易开始训练神经网络。许多图书和框架都以展示了如果采用 30 行代码解决您的数据问题，并以此而自豪。这给大家一个非常错误的印象，即这些东西是即插即用。常见的示例代码如下：

>>> your_data = # 导入数据
>>> model = SuperCrossValidator(SuperDuper.fit, your_data, ResNet50, SGDOptimizer)
# 由此征服世界这些

这些库和示例激活了我们大脑里面熟悉标准软件的部分 - 通常它们认为干净和抽象的 API 是可以轻易获得的。 比如，后面的功能可以采用如下调用：

>>> r = requests.get(''https://api.github.com/user'', auth=(''user'', ''pass''))
>>> r.status_code
200

　　这很酷！ 一个勇敢的开发人员已经承担了理解查询字符串，URL，GET / POST 请求，HTTP 连接等等的负担，并且在很大程度上隐藏了几行代码背后的复杂性。 这恰恰是我们熟悉和期待的。 不幸的是，神经网不是那样的。 它们不是 “现成的” 技术，第二个与训练 ImageNet 分类器略有不同。 我试图在我的帖子 “是的你应该理解反向传播” 中通过反向传播的例子说明这一点，并将其称为 “漏洞抽象”，但不幸的是情况更加可怕。 Backprop + SGD 并没有神奇地让你的网络正常工作。 Batch Norm 也不会神奇地使其收敛得更快。RNN 不会神奇地让你 “插入” 文本。 只是因为你可以将你的问题采用增强学习来建模，并不意味着你应该如此。 如果您坚持使用该技术而不了解其工作原理，则可能会失败。 这让我想到......

2. 神经网络训练无声地失败

当您写错或错误配置代码时，您通常会遇到某种异常。你输入了一个整数，但这本来应该是一个字符串的！某函数只需要 3 个参数！导入失败！键值不存在！两个列表中的元素个数不相等。此外，通常不会为特定功能创建单元测试。

解决了这些问题，也只是训练神经网络的开始。一切都可以在语法上正确，但整个训练没有妥善安排，而且很难说清楚。 “可能的错误面” 是很大的，逻辑（与语法相反）层面的问题，并且对单元测试非常棘手。例如，在数据增强期间左右翻转图像时，您可能忘记翻转数据标签。您的网络仍然可以（令人震惊地）工作得非常好，因为您的网络可以在内部学习检测翻转的图像，然后左右翻转其预测。或许你的自回归模型会因为一个偶然错误而意外地将它想要预测的东西作为输入。或者你想要修剪你的梯度但是修剪了损失，导致在训练期间异常样本被忽略。或者您从预训练模型初始化了您的权重，但没有使用原始均值。或者你只是用错了正则化权重，学习率，衰减率，模型大小等。因此，错误设置的神经网络只有在你运气好的时候才会抛出异常；大部分时间它会训练，但默默地输出看起来有点糟糕的结果。

因此，“快速和激烈” 方法训练的神经网络并不能发挥其作用，我们只会遭受其痛苦。 现在，痛苦是让神经网络运作良好的一个非常自然的过程，但可以通过对训练过程中的所有细节了然于胸来减轻训练过程的折磨。 在我的经验中，与深度学习成功最相关的品质是耐心和对细节的关注。

秘方

鉴于上述两个事实，我已经为自己开发了一个特定的过程，使我能够将神经网络应用于新问题。稍后我会竭力描述如何做到的。 你会发现它非常重视上述两个原则。 特别是，它遵循从简单到复杂的规律，并且在每一步我们对将要发生的事情做出具体假设，然后通过实验验证它们或进行检查直到我们发现了问题。 我们试图防止的是同时引入了许多复杂 “未经验证的” 问题，这必然会导致需要花很多时间去查找的错误 / 错误配置。 如果编写您的神经网络代码就像训练一样，您需要使用非常小的学习速率并猜测，然后在每次迭代后评估整个的测试集。

1. 了解你的数据

训练神经网络的第一步是根本不是接触任何神经网络代码，而是从彻底检查数据开始。这一步至关重要。我喜欢花费大量时间（以小时为单位）扫描数千个示例，了解它们的分布并寻找模式。幸运的是，你的大脑非常擅长这一点。有一次我发现了数据中包含重复的例子。另一次我发现了错误的图像 / 标签对。我通常会寻找不均衡的数据，也会关注自己对数据的分类过程，这些过程暗示了我们最终会尝试的各种架构。例如 - 我们需要局部特征还是全局上下文？数据有多少变化，这些变化采取什么形式？什么变化是假的，是可以预处理的？空间位置是否重要，或者我们是否想要将其平均化？细节有多重要，我们可以在多大程度上对图像进行下采样？标签有多少噪声？

此外，由于神经网络实际上可以看作压缩 / 编译的数据集，因此您将能够查看网络的（错误）预测并了解它们的来源。如果你的网络给你的预测看起来与你在数据中看到的内容不一致，那么就会有所收获。

一旦获得定性的感知，编写一些简单的代码来搜索 / 过滤 / 排序也是一个好主意，无论你能想到什么（例如标签的类型、大小、数量等），并可视化它们的分布，和沿任何坐标轴的异常值。异常值尤其能揭示数据质量或预处理中的一些错误。

2. 设置端到端的训练 / 评估框架 + 获得基准

当我们了解了数据之后，我们可以采用我们超级精彩的多尺度 ASPP FPN ResNet 训练牛 X 的模型吗？ 答案是不。 这是一条充满痛苦的道路。 我们的下一步是建立一个完整的训练 + 评估框架，并通过一系列实验验证其正确性。在这个阶段，最好选择一些你能正确使用的简单模型 - 例如 线性分类器，或非常小的 ConvNet。 我们希望对其进行训练，可视化损失，任何其他指标（例如准确度），模型预测，并在此过程中使用明确的假设进行一系列实验。

这个阶段的需要注意的地方和建议主要包括：

• 设置固定的随机种子。始终使用固定的随机种子，以保证当您运行两遍代码两次时，可以获得相同的结果。这消除了随机因素，并将帮助您保持理智。
• 简化。确保禁用任何不必要的尝试。例如，在此阶段肯定要关闭任何数据扩展。数据扩充是一种我们可能在以后合并的正规化策略，但是此刻它也可能引入一些愚蠢错误。
• 在评估时尽量准确。在绘制测试损失时，对整个（大）测试集进行评估。不要只是在批量上绘制测试损失，然后依靠在 Tensorboard 中平滑它们。我们追求正确，并且非常愿意放弃保持理智的时间。
• 验证初始的损失。验证您的损失函数是否以正确的损失值开始。例如。如果正确初始化最后一层，则应在初始化时查看 softmax 上的 - log（1 /n_classes），这默认值可以为 L2 回归，Huber 损失等。
• 正确初始化。正确初始化每层权重。例如，如果你正在回归一些平均值为 50 的值，那么将偏差初始化为 50。如果你有一个正负样本比例为 1:10 的不平衡数据集，请在你的数据上设置偏差，使网络初始化时就可以预测为 0.1。正确设置这些将加速收敛并消除 “曲棍球棒” 损失曲线，因为在最初的几次迭代中，您的网络基本上只是学习偏差。
• 人工评测。设置损失之外，人工可以查验和解释的的指标（例如准确性）。尽可能评估您自己（人类）的对该问题的准确性并与之进行比较。或者，对测试数据进行两次标注，并且对于每个示例，分别标注预测值和真实值。
• 指定和数据无关的标准。训练一个独立于输入数据的标准，（例如，最简单的方法是将所有输入设置为零）。这应该比实际插入数据时更糟糕，而不会将其清零。这可以吗？即你的模型是否学会从输入中提取任何信息？
• 在少量数据上过拟合。看看模型是否能够在仅仅少数几个数据（例如少至两个）上过拟合。为此，我们增加了模型的容量（例如添加层或过滤器）并验证我们可以达到可实现的最低损失（例如零）。我还想在同一个图中同时显示标签和预测，并确保一旦达到最小损失，它们就会完美重合。如果他们不这样做，那么某个地方就会出现一个错误，我们无法继续下一个阶段。
• 验证减少训练损失。在这个阶段，你希望模型在数据集上欠拟合，因为你正在使用玩具模型。尝试稍微增加容量，看看您的训练损失是否应该下降？
• 在训练模型前再次查看数据。在您 y_hat = model（x）（或 tf 中的 sess.run）之前，查看数据是否正确。也就是说 - 您希望确切地了解将要网络中的内容，将原始张量的数据和标签解码并将其可视化。这是唯一的 “事实来源”。我无法计算这个多少次节省了我的时间，并揭示了数据预处理和扩充中的问题。

• 预测过程动态可视化。我喜欢在训练过程中对固定测试数据上的预测结果进行可视化。这些预测如何动态发生的，将为您提供令人难以置信的直觉，使您能够了解训练的进展情况。很多时候，如果网络以某种方式摆动太多，显示出不稳定性，就有可能感觉网络难以拟合您的数据。学习率非常低或非常高，抖动量也很明显。

• 使用反向传递来绘制依赖关系。您的深度学习代码通常包含复杂的，矢量化的和广播的操作。我遇到的一个相对常见的错误是人们弄错了（例如他们使用视图而不是某处的转置 / 置换）并且无意中混合了 batch 的维度信息。不巧的是，您的网络通常仍然可以正常训练，因为它会学会忽略其他示例中的数据。调试此问题（以及其他相关问题）的一种方法是将某个示例的损失设置为 1.0，将反向传递一直运行到输入，并确保仅在该样本上获得非零梯度。更一般地说，梯度为您提供了网络中数据的依赖关系信息，这对调试很有用。
• 设置一个特例。这是一个更通用的编码技巧，但我经常看到人们因为贪心，从头开始编写相对一般的功能，而导入 bug。我喜欢为我现在正在做的事情编写一个专门的函数，让它工作，然后在确保我得到相同的结果再将其重构成一般功能的函数。这通常适用于矢量化代码，其中我几乎总是首先写出完全循环的版本，然后一次一个循环地将它转换为矢量化代码。

A Self-Organized Computer Virus Demo in C

A Program that can modify herself and copy herself to somewhere else and execute it. Just like gene-transformable virus.

Here''s the sample code.

#include "stdio.h"
#include "windows.h"

#define MAX_LOOP 10000

int main(int argc, const char** argv)
{
	/*** THE VIRUS LOGIC PART ***/
	//GENE_MARK
	int a = 0;

	printf("GENE PRINT:%d\n", a);
	/*** THE VIRUS LOGIC PART ***/


	// FILE NAME
	char file_name[30] = "test";

	// MAKE A COPY OF HERSELF
	FILE* source = fopen("test.c", "r");
	FILE* descendant = fopen("descendant.c", "w+");
	
	printf("SOURCE FILE OPEN RESULT IS : %d \n", (int)source);
	printf("DESCENDANT FILE CREATED: %d \n", (int)descendant);
	
	if(descendant==NULL)
	{
		printf("ERROR ON CREATING DESCENDANT.\n");
		return -1;
	}
	char buff[100] = {0};
	// REPLACE GENE MARK PROGRAM
	char letter = 0;
	
	// GENE LINE
	int idx = 0;
	int loop = 0;
	
	int buff_idx = 0;
	
	while(!feof(source))
	{
		// ALARM
		if(loop>MAX_LOOP)
			break;
		
		loop ++;
		
		fread(&letter, sizeof(char), 1, source);
		
		buff[buff_idx] = letter;
		buff_idx ++;
		
		if(letter==''\n'')
		{
			if(idx==9)
			{
				// TRANSFORM GENE
				memset(buff, 0, 100);
				buff_idx = 0;
				strcat(buff, "int a = 1;\n");
			}
			fwrite(buff, sizeof(char), strlen(buff), descendant);
			// CLEAR BUFFER
			memset(buff, 0, 100);
			buff_idx = 0;
			idx ++;
		}
	}
	
	// DEAL WITH LEFT LETTERS IN BUFFER
	if(strlen(buff)>0)
	{
		strcat(buff, "\n");
		fwrite(buff, sizeof(char), strlen(buff)+1, descendant);
	}
	
	// CLOSE ALL FILES
	fclose(source);
	fclose(descendant);
	
	// until the descendant file is written over
	
	/*** COMPILE HERSELF ***/
	char* source_file = "descendant.c";
	char* dest_file = "descendant.exe";
	char command[100] = {0}; 
	strcat(command, "gcc -o ");
	strcat(command, dest_file);
	strcat(command, " ");
	strcat(command, source_file);
	// COMPILATION
	system(command);
	/***********************/
	
	
	printf("COPYING MYSELF DONE.\n");
	
	printf("WAITING FOR NEXT INSTRUCTION...\n");
	char cmd = getchar();
	if(cmd==''Y'')
	{
		printf("BEGIN EXECUTE THE COPYFILE EXECUTION...\n");
//GENE_MARK	
		system("descendant.exe");
		printf("EXECUTION PROCESS IS ACTIVATED, TASK DONE. EXIT SYSTEM.");
	}
	else
		printf("YOU CHOOSE TO EXIT SYSTEM. BYE!");

	return 0;
}

If you have any suggestions or ideas, please feel free comment below, thanks!

Chapter 1: A Tour of Computer Systems

    A computer system consists of hardware and systems software that work together to run application programs.

    We begin our study of systems by tracing the lifetime of the hello program, from the time it is created by a programmer, until it runs on a system, prints its simple message, and terminates.

#include <stdio.h>

int main(void)
{
    printf("hello, world\n");
}

1.1 Information Is Bits + Context

    The hello.c program is stored in a file as a sequence of bytes, we shows the ASCII representation of the hello.c program:

    Files such as hello.c that consist exclusively of ASCII characters are known as text files.All other files are known as binary files.

    The representation of hello.c illustrates a fundamental idea: All information in a system--including disk files, programs stored in memory, user data stored in memory, and data transferred across a network--is represented as a bunch of bits.The only thing that distinguishes different data objects is the context in which we view them.

1.2 Programs Are Translated by Other Programs into Different Forms

    in order to run hello.c on the system, the individual C statements must be translated by other programs into a sequence of low-level machine-language instructions.These instructions are then packaged in a form called an executable object program and stored as a binary disk file.Object programs are also referred to as executable object files.

    we run hello.c in unix system:

lgtdeMacBook-Pro:~ lgt$ gcc -o hello hello.c
lgtdeMacBook-Pro:~ lgt$ ls hello*
hello	hello.c
lgtdeMacBook-Pro:~ lgt$ ./hello
hello, world

Preprocessing phase:The preprocessor(cpp) modifies the original C program according to directives that begin with the # character.For example, the #include<stdio.h> command in line 1 of hello.c tells the preprocessor to read the contents of the system header file stdio.h and insert it directly into the program text. The result is another C program, typically with the .i suffix.

Compilation phase:The compiler(cc1) translates the text file hello.i into the text file hello.s, which contains an assembly-language program.Each statement in an assembly-language program exactly describes one low-level machine-language instruction in a standard text form.Assembly language is useful because it provides a common output language for different compilers for different high-level languages.

Assembly phase: The assembler(as) translates hello.s into machine-language instructions, packages then in a form known as a relocatable object program, and stores the result in the object file hello.o.The hello.o file is a binary file whose bytes encode machine language instructions rather than characters.

Linking phase:The printf function resides in a separate precompiled object file called print.o, which must somehow be merged with our hello.o program. The linker(ld) handles this merging.The result is the hello file, which is an executable object file(or simply executable) that is ready to be loaded into memory and executed by the system.

1.3 It Pays to Understand How Compilation Systems Work

    how compilation system work: Optimizing program performance, Understanding link-time errors and Avoiding security holes.

1.4 Processors Read and Interpret Instructions Stored in Memory

1.4.1 Hardware Organization of a System

    To understand what happens to our hello program when we run it, we need to understand the hardware organization of a typical system:

Buses

    Running throughout the system is a collection of electrical conduits called buses that carry bytes of information back and forth between the components. Buses are typically designed to transfer fixed-sized chunks of bytes known as words.The number of bytes in a word(the word size) is a fundamental system parameter that varies across systems.

I/O Devices

    Input/output(I/O) devices are the system''s connection to the external world.

    Each I/O device is connected to the I/O bus by either a controller or an adapter. Controllers are chip sets in the device itself or on the system''s main printed circuit board(often called the mother board). An adapter is a card that plugs into a slot on the motherboard. Regardless, the purpose of each is to transfer information back and forth between the I/O bus and an I/O device.

Main Memory

    The main memory is a temporary storage device that holds both a program and the data it manipulates while the processor is executing the program. Physically, main memory consists of a collection of dynamic random access memory(DRAM) chips. Logically, memory is organized as a linear array of bytes, each with its own unique address(array index) starting at zero. In general, each of the machine instructions that constitute a program can consist of a variable number of bytes. The sizes of data items that correspond to C program variables vary according to type.

Processor

    The central processing unit(CPU) or simply processor, is the engine that interprets(or executes) instructions stored in main memory.At its core is a word-sized storage device(or register) called the program counter(PC). At any point in time, the PC points at (contains the address of) some machine-language instruction in main memory.

    From the time that power is applied to the system, until the time that the power is shut off, a processor repeatedly executes the instruction pointed at by the program counter and updates the program counter to point to the next instruction. A processor appears to operate according to a very simple instruction execution model, defined by its instruction set architecture. In this model, instructions execute in strict sequence, and executing a single instruction involves performing a series of steps. The processor reads the instruction from memory pointed at by the program counter(PC), interprets the bits in the instruction, performs some simple operation dictated by the instruction, and then updates the PC to point to the next instruction, which may or may not be contiguous in memory to the instruction that was just executed.

    There are only a few of these simple operations, and they revolve around main memory, the register file, and the arithmetic/logic unit(ALU). The register file is a small storage device that consists of a collection of word-sized registers, each with its own unique name. The ALU computes new data and address values. Here has some example:

Load: Copy a byte or a word from main memory into a register, overwriting the previous contents of the register.

Store: Copy a byte or a word from a register to a location in main memory, overwriting the previous contents of that location.

Operate: Copy the contents of two registers to the ALU, perform an arithmetic operation on the two words, and store the result in a register, overwriting the previous contents of that register.

Jump: Extract a word from the instruction itself and copy that word into the program counter(PC), overwriting the previous value of the PC.

1.4.2 Running the hello Program

    When we type the characters "./hello" at the keyboard, the shell program reads each one into a register, and then stores it in memory:

    When we hit the enter, the shell then loads the executable hello file by executing a sequence of instructions that copies the code and data in the hello object file from disk to main memory. The data include the string of characters "hello, world\n" that will eventually be printed out:

    Once the code and data in the hello object file are loaded into memory, the processor begins executing the machine-language instructions in the hello program''s main routine.

1.5 Caches Matter

    We know that processor is faster than main memory, and main memory is faster than disk when moving information from one place to another.

    To deal with the processor-memory gap, system designers include smaller faster storage devices called cache memories(or simply caches) that serve as temporary staging areas for information that the processor is likely to need in the near future.

An L1 cache on the processor chip holds tens of thousands of bytes and can be accessed nearly as fast as the register file. A larger L2 cache with hundreds of thousands to millions of bytes is connected to the processor by a special bus. It might take 5 times longer for the process to access the L2 cache than the L1 cache, but this is still faster than accessing the main memory. The L1 and L2 caches are implemented with a hardware technology known as static random access memory(SRAM).

1.6 Storage Devices Form a Hierarchy

    memory hierarchy:

1.7 The Operating System Manages the Hardware

    Back to our hello example. When the shell loaded and ran the hello program, and when the hello program printed its message, neither program accessed the keyboard, display, disk, or main memory directly. Rather, they  relied on the services provided by the operating system. We can think of the operating system as a layer of software interposed between the application program and the hardware. All attempts by an application program to manipulate the hardware must go through the operating system.

    The operating system has two primary purposes:(1) to protect the hardware from misuse by runaway applications, and (2) to provide applications with simple and uniform mechanisms for manipulating complicated and often wildly different low-level hardware devices.

    files are abstractions for I/O devices, virtual memory is an abstraction for both the main memory and disk I/O devices, and processes are abstractions for the processor, main memory, and I/O devices.

1.7.1 Processes

    A process is the operating system''s abstraction for a running program.

    The operating system keeps track of all the state information that the process needs in order to run. This state, which is known as the context, includes information such as the current values of the PC, the register file, and the contents of main memory. At any point in time, a uniprocessor system can only execute the code for a single process. When the operating system decides to transfer control from the current process to some new process, it performs a context switch by saving the context of the current process, restoring the context of the new process, and then passing control to the new process.

1.7.2 Threads

    threads running in the context of the process and sharing the same code and global data.

1.7.3 Virtual Memory

    Virtual memory is an abstraction that provides each process with the illusion that it has exclusive use of the main memory. Each process has the same uniform view of memory, which is known as its virtual address space.

1.7.4 Files

    A file is a sequence of bytes.

1.8 Systems Communicate with Other Systems Using Networks

    The network can be viewed as just another I/O device. With the advent of global networks such as the Internet, copying information from one machine to another has become one of the most important uses of computer systems:

    Returning to our hello example, we could use the familiar telnet application to run hello on a remote machine:

1.9 Important Themes

1.9.1 Concurrency and Parallelism

    we want computer to do more, and we want them to run faster. Both of these factors improve when the processor does more things at once. We use the term concurrency to refer to the general concept of a system with multiple, simultaneous activities, and the term parallelism to refer to the use of concurrency to make a system run faster.Parallelism can be exploited at multiple levels of abstraction in a computer system.

Thread-Level Concurrency

    We use multi-core processors and hyper threading to make a system to consist of multiple processors.

    Multi-core processors have several CPUs integrated onto a single integrated-circuit chip:

    Hyperthreading, sometimes called simultaneous multi-threading, is a technique that allows a single CPU to execute multiple flows of control.It involves having multiple copies of some of the CPU hardware, such as program counters and register files, while having only single copies of other parts of the hardware, such as the units that perform floating-point arithmetic.

COEN 146: Computer Networks

COEN 146: Computer Networks
Lab 3: TCP/IP Socket Programming

Objectives
1.To develop client/ server applications using TCP/IP Sockets
2.To write a C program to transfer file over TCP/IP Socket

TCP/IP Client/ Server[ http://beej.us/guide/bgnet/] 
The following figure shows the steps taken by each program:
On the client and server sides:

The socket() system call creates an unbound socket in a communications domain, and return a file descriptor that can be used in later function calls that operate on sockets.

int sockfd = socket(domain, type, protocol)
●sockfd: socket descriptor, an integer (like a file-handle)
●domain: integer, communication domain e.g., AF_INET (IPv4 protocol) , AF_INET6 (IPv6 protocol), AF_UNIX (local channel, similar to pipes)
●type: communication type
SOCK_STREAM: TCP (reliable, connection oriented)
SOCK_DGRAM: UDP (unreliable, connectionless)
SOCK_RAW (direct IP service)
●protocol: This is useful in cases where some families may have more than one protocol to support a given type of service. Protocol value for Internet Protocol (IP), which is 0. This is the same number which appears on protocol field in the IP header of a packet.

include <sys/socket.h>

...
...if ((server_fd = socket(AF_INET, SOCK_STREAM, 0)) < 0) 
{
    perror(“cannot create socket”); 
    return 0; 
}

On the server side:

After creation of the socket, bind() system call binds the socket to the address and port number specified in addr(custom data structure). In the example code, we bind the server to the localhost, hence we use INADDR_ANY to specify the IP address.

int bind (int sockfd, const struct sockaddr *addr, socklen_t addrlen);
●addr: Points to a sockaddr structure containing the address to be bound to the socket. The length and format of the address depend on the address family of the socket.
●addrlen: Specifies the length of the sockaddr structure pointed to by the addr argument. 

The listen() function puts the server socket in a passive mode, where it waits for the client to approach the server to make a connection. 

int listen(int sockfd, int backlog);
●backlog: defines the maximum length to which the queue of pending connections for sockfd may grow. If a connection request arrives when the queue is full, the client may receive an error with an indication of ECONNREFUSED.

The accept() system call extracts the first connection request on the queue of pending connections for the listening socket (sockfd), creates a new connected socket, and returns a new file descriptor referring to that socket. At this point, connection is established between client and server, and they are ready to transfer data.

int new_socket= accept(int sockfd, struct sockaddr addr, socklen_t addrlen);

On the client side:

The connect() system call connects the socket referred to by the file descriptor sockfd to the address specified by addr. Server’s address and port is specified in addr.

int connect(int sockfd, const struct sockaddr *addr, socklen_t addrlen);

Read and write over socket:

     bzero(buffer, 256);
     n = read(newsockfd, buffer, 255);
     if (n < 0) error("ERROR reading from socket");
     printf("Here is the message: %s\n", buffer);

This code initializes the buffer using the bzero() function, and then reads from the socket. 
Note that the read call uses the new file descriptor, the one returned by accept(), not the original file descriptor returned by socket(). 

Note also that the read() will block until there is something for it to read in the socket, i.e. after the client has executed a write(). It will read either the total number of characters in the socket or 255, whichever is less, and return the number of characters read

    n = write(newsockfd, "I got your message", 18);
         if (n < 0) error("ERROR writing to socket");

Once a connection has been established, both ends can both read and write to the connection. Naturally, everything written by the client will be read by the server, and everything written by the server will be read by the client. This code simply writes a short message to the client. The last argument of write is the size of the message.

Structures:

Address format
An IP socket address is defined as a combination of an IP interface address and a 16-bit port number. The basic IP protocol does not supply port numbers, they are implemented by higher level protocols like UDP and TCP. On raw sockets sin_port is set to the IP protocol.

struct sockaddr_in {
    sa_family_t    sin_family; / address family: AF_INET /
    in_port_t      sin_port;   / port in network byte order /
    struct in_addr sin_addr;   / internet address /
};

/ Internet address. /
struct in_addr {
    uint32_t       s_addr;     / address in network byte order /
};

This is defined in netinet/in.h

sin_family is always set to AF_INET. 

sin_port contains the port in network byte order. The port numbers below 1024 are called privileged ports (or sometimes: reserved ports). Only privileged processes) may bind to these sockets.

sin_addr is the IP host address. 

s_addr member of struct in_addr contains the host interface address in network byte order. 
in_addr should be assigned one of the INADDR_* values (e.g., INADDR_ANY) or set using the inet_aton, inet_addr, inet_makeaddr library functions or directly with the name resolver (see gethostbyname).

INADDR_ANY allows your program to work without knowing the IP address of the machine it was running on, or, in the case of a machine with multiple network interfaces, it allowed your server to receive packets destined to any of the interfaces. 

INADDR_ANY has the following semantics: When receiving, a socket bound to this address receives packets from all interfaces. For example, suppose that a host has interfaces 0, 1 and 2. If a UDP socket on this host is bound using INADDR_ANY and udp port 8000, then the socket will receive all packets for port 8000 that arrive on interfaces 0, 1, or 2. If a second socket attempts to Bind to port 8000 on interface 1, the Bind will fail since the first socket already “owns” that port/interface.
Example:
serv_addr.sin_addr.s_addr = htonl (INADDR_ANY);
●Note: "Network byte order" always means big endian. "Host byte order" depends on architecture of host. Depending on CPU, host byte order may be little endian, big endian or something else. 
●The htonl() function translates a long integer from host byte order to network byte order.

To bind socket with localhost, before you invoke the bind function, sin_addr.s_addr field of the sockaddr_in structure should be set properly. The proper value can be obtained either by 

my_sockaddress.sin_addr.s_addr = inet_addr("127.0.0.1")
or by 
my_sockaddress.sin_addr.s_addr=htonl(INADDR_LOOPBACK);
To convert an address in its standard text format into its numeric binary form use the inet_pton() function. The argument af specifies the family of the address. 

define _OPEN_SYS_SOCK_IPV6

include <arpa/inet.h>

int inet_pton(int af, const char src, void dst);

Recap - File transfer:
●Binary file: jpg, png, bmp, tiff etc.
●Text file: txt, html, xml, css, json etc.

You may use functions or system calls for file transfer. C Function connects the C code to file using I/O stream, while system call connects C code to file using file descriptor.
●File descriptor is integer that uniquely identifies an open file of the process.
●I/O stream sequence of bytes of data.

A Stream provides high level interface, while File descriptor provide a low-level interface. Streams are represented as FILE * object, while File descriptors are represented as objects of type int.

C Functions to open and close a binary/text file
fopen(): C Functions to open a binary/text file, defined as:
FILE fopen(const char file_name, const char *mode_of_operation);
where:
●file_name: file to open
●mode_of_operation: refers to the mode of the file access, For example:- r: read , w: write , a: append etc
●fopen() return a pointer to FILE if success, else NULL is returned
●fclose(): C Functions to close a binary/text file.

fclose(): C Functions to close a binary/text file, defined as:
fclose( FILE *file_name);
Where:
●file_name: file to close
●fclose () function returns zero on success, or EOF if there is an error

C Functions to read and write a binary file
fread(): C function to read binary file, defined as:
fread(void ptr, size_t size, size_t count, FILE stream);
where:
●ptr- it specifies the pointer to the block of memory with a size of at least (size*count) bytes to store the objects.
●size - it specifies the size of each objects in bytes.
●count: it specifies the number of elements, each one with a size of size bytes.
●stream - This is the pointer to a FILE object that specifies an input stream.
●Returns the number of items read

fwrite (): C function to write binary file, defined as:
fwrite (void ptr, size_t size, size_t count, FILE stream);
where:
●Returns number of items written
●*arguments of fwrite are similar to fread. Only difference is of read and write.

For example:
To open "lab3.dat" file in read mode then function would be:
FILE* demo; // demo is a pointer of type FILE
char buffer[100]; // block of memory (ptr)
demo= fopen("lab3.dat", "r"); // open lab3.dat in read mode
fread(&buffer, sizeof(buffer), 1, demo); // read 1 element of size = size of buffer (100)
fclose(demo); // close the file

C Functions to read and write the text file.
fscanf (): C function to read text file.
fscanf(FILE ptr, const char format, ...)
Where:
●Reads formatted input from the stream.
●Ptr: File from which data is read.
●format: format of data read.
●returns the number of input items successfully matched and assigned, zero if failure

fprintf(): C function to write a text file.
fprintf(FILE ptr, const char format, ...);
Where:
●*arguments similar to fscanf ()

For example:
FILE *demo; // demo is a pointer of type FILE
demo= FILE *fopen("lab3.dat", "r"); // open lab3.dat in read mode
/* Assuming that lab3.dat has content in below format
City
Population
….
*/
char buf[100]; // block of memory
fscanf(demo, "%s", buf); // to read a text file
fclose(demo); // close the file
*to read whole file use while loop

System Call to open, close, read and write a text/binary file.
open(): System call to open a binary/text file, defined as:
open (const char* Path, int flags [, int mode ]);
Where:
●returns file descriptor used on success and -1 upon failure
●Path :- path to file
●flags :- O_RDONLY: read only, O_WRONLY: write only, O_RDWR: read and write, O_CREAT: create file if it doesn’t exist, O_EXCL: prevent creation if it already exists

close(): System call to close a binary/text file, defined as:
close(int fd);
where:
●return 0 on success and -1 on error.
●fd : file descriptor which uniquely identifies an open file of the process

read(): System call to read a binary/text file.
read (int fd, void* buf, size_t len);
where:
●returns 0 on reaching end of file, -1 on error or on signal interrupt
●fd: file descriptor
●buf: buffer to read data from
●len: length of buffer

write(): System call to write a binary/text file.
write (int fd, void* buf, size_t len);
where:
●*arguments and return of write are similar to read(). 

For example:
int fd = open("lab3.dat", O_RDONLY | O_CREAT); //if file not in directory, file is 
created
Close(fd);

Implementation steps:
Step 1.[30%] Write a C program for a TCP server that accepts a client connection for file transfer. 
Step 2.[25%] Write a C program for a TCP client that connects to the server. In this case
a.The client connects to the server and request a file to download from the server. 
b.The server accepts the connection and transfers the file to the client

Step 3.Compile and run. Note: you may use the IP address 127.0.0.1 (loop back IP address) for a local host, i.e. both of your client and server run on the same machine. 

[20%] Demonstrate your program to the TA:
a.Your client and server on your same machine
b.Your client and your classmate’s server IP address. You may to discuss with the TA if you run into access problems  

Multiple Clients – Concurrent Server 
In general, a TCP server is designed as a concurrent server to server multiple clients. This means when a client sends a request for a file transfer, the sever accepts the connection request and spawns a thread to handle this transfer on the connection descriptor. The server will then continue in a loop listening for another client connection request to handle another file transfer.

Step 4.[20%] Write a C program for a concurrent TCP server that accepts and responds to multiple client connection requests, each requesting a file transfer. Modify your TCP server in Step 1 so that when the server accepts a connection from a client it spawns a separate thread to handle this specific client connection for file transfer. 

Note: You will have several threads (at the same time) running on the server transferring copies of src.dat files to clients that each will save at their destination as – dst.dat file (possibly needs to be numbered differently on the same host).

[5%] Demonstrate to the TA, multiple clients making file transfer request to the server and that the server makes multiple transfers at the same time. Make N = 5. Upload your source code to Camino. 

Note: To be able to see 5 threads handling connections at the same time, you may need to introduce a delay of a few second in the process of data transfer to make it visible. This is due to the fact completing thread file transfer takes a fraction of a millisecond if not a microsecond. 

Requirements to complete the lab
1.Demo to the TA correct execution of your programs [recall: a successful demo is 25% of the grade]
2.Submit the source code of your program as .c files on Camino

Please start each program with a descriptive block that includes minimally
WX：codinghelp

关于COMP1047 Part-2: Computer Networks的介绍已经告一段落，感谢您的耐心阅读，如果想了解更多关于A Recipe for Training Neural Networks [中文翻译，part 1]、A Self-Organized Computer Virus Demo in C、Chapter 1: A Tour of Computer Systems、COEN 146: Computer Networks的相关信息，请在本站寻找。