How to build a Beowulf Cluster Applied to Computational

Transcrição

How to build a Beowulf Cluster Applied to
Computational Electromagnetic
Carlos Henrique da Silva Santos
Leonardo André Ambrosio
Hugo Enrique Hernández Figueroa
Department of Microwaves and Optics (DMO)
School of Electrical and Computer Engineering (FEEC)
State University of Campinas (UNICAMP)
Summary
•
•
1.
2.
3.
4.
•
•
Motivation;
Objectives;
Preview: Operation Systems and Network concepts;
Installing Linux Fedora distribution;
Configuring a computer cluster;
HPC cluster applications in electromagnetism;
Conclusions;
Main references;
How to Build a Cluster
Carlos Henrique
2
Motivations
• Different concepts over High Performance Computing (HPC) applied to
Computational Electromagnetism;
• Low computational costs solutions for HPC;
• New computational technologies and resources to build a HPC cluster;
• Some large computational electromagnetic application.
Carlos Henrique
3
Objectives
• Presenting some computational concepts:
• Operating system layers
• Network structures;
• Linux distributions and services (NFS and SSH);
• Sophisticated and easy ways to build a computer cluster, focusing on data
security;
• How to configure services and connections.
Carlos Henrique
4
1st Part - Concepts
Operating System
Linux
Network
Carlos Henrique
5
Operating Systems
Kernel
Hardware
Kernel
System Calls
Program
Carlos Henrique
6
Operating System: Structure
•
Memory management;
•
Disk and file systems;
•
Networking;
•
Security:
• Internal: Programs running locally;
• External: Network connections;
•
Graphical user interface;
•
Device Driver;
•
Process management.
Firewall
Carlos Henrique
7
Operating System: Process
• Process:
• A program in execution;
• An instance of a program running in a computer;
• The entity that can be assigned and executed on a processor (program
code and a dataset);
Identifier
State
Priority
Program counter
Memory pointers
Context data
I/O Status information
Accounting information
.
.
.
Carlos Henrique
8
Modern Unix System
coff
a.out
elf
NFS
Exec
switch
File
mappings
FFS
virtual
memory
framework
Device
mappings
vnode/vfs
interface
Anonymous
mappings
RFS
Common
facilities
Disk driver
Time sharing
process
block
device
switch
Scheduler
framework
Tape driver
System
processes
Streams
Network
driver
s5fs
Carlos Henrique
tty
driver
9
Why Linux is necessary?
• Linux is a free operating system;
• It was concepted by Linus Torvalds to attempt his necessities, without
intentions to get money;
• The development style adopted is the collective help, where someone
manages the colective efforts of a group to improve the system;
• Millions of people are gratuitously contributing to develop Linux, for pleasure
or to have a better operating system.
Carlos Henrique
10
Operating System: Linux
• It is a multiprocessor and multiuser system, which means that many users
have the possibility to execute many processes into the same computer using
Linux in the same time;
• Free and open source software, basing on the GPL. Furthermore, most of the
computational tools for this operational system are free too;
• It has some different Kernel to provide better solutions depending on the
application;
• It is one of the best operational system for network and HPC applications;
• Free tutorial, manual and help on the Internet.
Carlos Henrique
11
Networking Application: Computer Cluster
0
Carlos Henrique
1
12
Network: Services, Protocols and Layers
• Protocols: are related to the packets sent between peer entities on different
machines;
• Services: is a set of primitives (operations) that a layer provides to the layer
above it. In other words, is related to the interface between layers.
Machine 1
Machine 2
Layer k+1
Layer k+1
Service provided by layer k
Protocol
Layer k
Layer k-1
Layer k-1
Layer k
Carlos Henrique
13
Network: OSI Reference Model layers
7
Application
Application protocol
Application
APDU
Presentation
PPDU
Session
SPDU
Transport
TPDU
Interface
6
Presentation
5
Session
4
Transport
Presentation protocol
Session protocol
Transport protocol
Communication subnet boundary
Internet subnet protocol
3
Network
Network
Network
Network
Packet
2
Data link
Data link
Data link
Data link
Data link
1
Physical
Physical
Physical
Physical
Bit
Carlos Henrique
14
Network: Open Systems Interconnection (ISO-OSI) Reference Model
•
Deals with connecting open system, which are open for communication with
other systems;
•
OSI model has seven layers, based on:
1. A layer should be created where a different abstraction is needed;
2. Each layer should perform a well-defined function;
3. Each layer function should be defined in order to provide an
international standardization protocol;
4. The layer boundaries should be chosen to minimize the information
flow across the interfaces;
5. The number of layer should be large enough that distinct functions
and small enough that the architecture does not become widely.
Carlos Henrique
15
2nd Part – Installing Fedora RedHat
Carlos Henrique
16
Choosing installation version
• i386
• Intel x86-compatible processors, including Intel Pentium and PentiumMMX, Pentium Pro, Pentium-II, Pentium-III, Celeron, Pentium 4, Xeon,
Core Duo, and Centrino/Centrino Duo; VIA C3/C3-m and Eden/Eden-N;
and AMD Athlon, AthlonXP, Duron, AthlonMP, Sempron, and Turion
• ppc
• PowerPC processors, such as those found in Apple Power Macintosh, G3,
G4, and G5, and IBM pSeries systems
• x86_64
• 64-bit AMD processors such as Athlon64/FX/X2, Turion64, Opteron; and
Intel 64-bit processors such as EM64T
Carlos Henrique
17
Pre-installation
Carlos Henrique
18
Installation: System properties
Carlos Henrique
19
Installation: Operating System Resource
Carlos Henrique
20
3rd Part – Cluster
• Configuring Linux
• Network
• SSH
• NFS
• Installing MPI Libraries
Carlos Henrique
21
TCP: Transport Control Protocol
• Transport Layer at OSI Model
• It is connection oriented, which means that all problems no solved in IP
level will be solved here;
• Some problems that must be solved by TCP:
• Lost of packages or destructions occurs
transmission;
• Package expedition out of order or duplicated.
Carlos Henrique
during the
22
• The TCP specify the data package format and patterns recognized in
reliable changes between two computers, like the procedures that were
applied in computers to ensure the data communication;
• Procedures to be considered:
• Distinguishing between multiples destinies in a determined machine;
• Error recovering, like some lost packages or duplicated.
Carlos Henrique
23
• Available multiple programs runs in one machines communicate;
• Multiplex the TCP input traffic between the applications programs;
• Uses numerical port to identify the last destination in a machine. Each port
is identified by a low integer number;
• Example: 128.10.2.3.25 specifies the TCP port number 25, in the machine
with IP address 128.10.2.3.
Carlos Henrique
24
IP: Internet Protocol
Network Configuration - Structure
http://www.informabr.com.br/ipsubnet.htm
Carlos Henrique
25
Linux Network Configuration: shell Mode
• Identify network device in use;
• Open the file:
• /etc/sysconfig/network-scripts/ifcfg-ethx
DEVICE=ethx
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
IPADDR=10.0.0.100
NETWORK=10.0.0.0
BROADCAST=10.0.0.255
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
DNS1=10.0.0.1
#
#
#
#
#
#
#
#
#
#
Identify the device
Type of boot (none, static ou dhcp)
Starting on machine boot
Type of network
IP address of the machine at intranet
Network Masquerade
Range of IP address in the network
Defines the subnet
Package destination to network exit
Import
the
DNS
specified
on
/etc/resolv.conf
Carlos Henrique
26
Shell Script – Network Address Translation
modprobe
modprobe
modprobe
modprobe
modprobe
modprobe
modprobe
echo 1 >
echo 1 >
echo 1 >
echo 1 >
for f in
echo
done
for f in
echo
done
for f in
echo
done
for f in
echo
done
iptables
ip_conntrack
ip_conntrack_ftp
ip_tables
ipt_state
iptable_nat
ip_nat_ftp
ipt_limit
/proc/sys/net/ipv4/ip_forward
/proc/sys/net/ipv4/tcp_syncookies
/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
/proc/sys/net/ipv4/icmp_ignore_bogus_error_responses
/proc/sys/net/ipv4/conf/*/rp_filter; do
1 >$f
/proc/sys/net/ipv4/conf/*/accept_redirects; do
0 >$f
/proc/sys/net/ipv4/conf/*/send_redirects; do
0 >$f
/proc/sys/net/ipv4/conf/*/accept_source_route; do
0 >$f
-t nat -A POSTROUTING -o eth0 -j MASQUERADE
Carlos Henrique
27
Main Network Services Requested
• FTP/SFTP (File Transfer Protocol/SSH File Transfer Protocol): to share
and access files in different machines or networks;
• SSH: Secure Shell: remote access control;
• NFS (Network File System): sharing files in a network
Carlos Henrique
28
SSH: Secure Shell
• O SSH has been defined on IANA beyond the RFC 4250;
• A free and open-source available is OpenSSH, which supports the versions
1.3, 1.5 e 2.0 of the SSH protocol;
• Shell command:
ssh cepofslave
• The first time that a connection for a different is required, the following
message is exhibit:
The authenticity of host ‘cepofslave’ can’t be established.
DSA key fingerprint is 95:90:3a:3a:bc:f3:9a:9b:01:5d:b3:07:38:e2:11:0c.
Are you sure you want to continue connecting (yes/no)?
Carlos Henrique
29
NFS: Network File System
• It is a model to structure data file distribution in some servers, mounting a
virtual directory.;
• Making possible to users to centralize their folders in a server;
• These tools is important in High Performance Computing structures that
access disc. Computer cluster is an evident example.
Carlos Henrique
30
• To allow client access in NFS servers the following Daemons are
necessary:
• nfsd: to receive NFS clients requests;
• mountd: daemon to mount NFS partitions and to execute nfsd
requests;
• portmap: allows to NFS clients dicover which NFS server port is
using.
Carlos Henrique
31
• A client can connect a shared data file, he needs of permission to do it. For
this reason, is important to certify if he is allowed in the file /etc/exports;
• In the /etc/exports each line represents one file system to be exported. A
remote system can be specify just only one time by the file system, and
only one standard input can be defined.
IP range identification
/home/cluster 10.0.0.0/255.255.255.0(rw,no_root_squash)
Identifying IPs
/home/cluster 10.0.0.100(rw,no_root_squash)
/home/cluster 10.0.0.101(rw,no_root_squash)
Carlos Henrique
32
Configure Cluster: Server
Creating one user called cepof
Step 0
adduser cepof
passwd cepof
/etc/exports
NFS
Step 1
/home/cepof 10.0.0.101 (rw, no_root_squash)
/etc/hosts
Step 3
10.0.0.101
10.0.0.102
10.0.0.103
cepofslave1
cepofslave2
cepofslave3
/etc/hosts.equiv
Step 4
cepofslave1
cepofslave2
cepofslave3
Carlos Henrique
33
Configure Cluster: Server
Reboot NFS service
Step 5
service nfs restart
Change to cepof user
Step 6
su cepof
Creating the cryptographic key to access
Step 7
ssh-keygen -t dsa
Creating authorization file
Step 8
cat .ssh/id_dsa.pub > authorized_keys
Permission of the directory .ssh
Step 9
chmod 700 .ssh
File access permission
Step 10
chmod 644 .ssh/authorized_keys
Carlos Henrique
34
Configure Cluster: Slave nodes
Creating cepof user
Step 0
adduser cepof
Passwd cepof
/etc/hosts
Step 1
Step 2
10.0.0.101
10.0.0.102
10.0.0.103
cepofslave1
cepofslave2
cepofslave3
/etc/fstab
10.0.0.1:/home/cluster /home/cluster nfs
Carlos Henrique
exec,dev,suid,rw 1 1
35
Configure Cluster: Slave nodes
Step 3
Step 4
Step 5
Step 6
Step 7
Step 8
FTP connection on the Server
sftp [email protected]
Getting the public key
get .ssh/id_dsa.pub
Ending connection
exit
Creating the authorization key file
cat .ssh/id_dsa.pub > authorized_keys
Permissions of the .ssh directory
chmod 700 .ssh
Permissions of the authorization file
chmod 644 .ssh/authorized_keys
Carlos Henrique
36
Example: “Hello World”
#include <unistd.h>
#include <stdio.h>
#include "mpi.h"
int main(int argc, char ** argv)
{
int my_rank;
int p;
int source;
int dest;
int silen = 128;
int gherr;
char hname[128];
char message[800];
MPI_Status status;
int tag = 50;
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
MPI_Comm_size(MPI_COMM_WORLD, &p);
gherr = gethostname( hname, silen);
if (my_rank != 0) {
sprintf(message, "Greetings from process %d on %s!", my_rank, hname);
dest = 0;
MPI_Send(message, strlen (message)+1, MPI_CHAR, dest,
tag, MPI_COMM_WORLD);
} else {
printf ("Messages received by process %d on %s.\n\n", my_rank, hname);
for (source = 1; source < p; source++) {
MPI_Recv(message, 800, MPI_CHAR, source, tag,
MPI_COMM_WORLD, &status);
printf("%s\n", message);
}
}
MPI_Finalize();
return 0;
}
Carlos Henrique
37
Parallel Process Schematic
Process
Process00(Mater
(MaterNode)
Node)
Process
Process11
Process
Process22
Carlos Henrique
Process
Process33
38
Creating boot configuration file
• vi lamhosts
• Add every IP or hosts of the cluster machines
10.0.0.1
10.0.0.101
10.0.0.102
LAM 7.1.1/MPI 2 C++/ROMIO - Indiana University
• lamboot –v lamhosts
• mpiCC hello.c –o hello
• mpirun C hello
Carlos Henrique
n-1<13229> ssi:boot:base:linear: booting n0 (10.0.0.1)
n-1<13229> ssi:boot:base:linear: finished
39
4th Part – Applications of HPC
Electromagnetic Applications
Carlos Henrique
40
Cluster Application: Human Head SAR Simulation
Slice 1
Slice 2
Slice 4
y
x
Slice 3
Air
Skin
Bone
Blood
Brain
Muscle
Temporal Muscle
Slice 5
Ω
Carlos Henrique
Ω1
Ω2
Ω3
41
Cluster Application: Human Head SAR Simulation
• SAR calculus considering energy and mass;
σ
SAR =
Ei
2ρ
2
=
Ji
2
2 ρσ
[ mW / g ]
• Applying the Poynting vector
SAR =
∂ ∂W ∂ ∂W
=
[ mW / g ]
∂t ∂m ∂t ∂V
• Local average of the SAR in terms of the simulation time (Nmax)
σ 1
SAR =
2 ρ N max
Carlos Henrique
∑ [E
2
x
]
+ E y2 [ mW / g ]
42
Cluster Application: Human Head SAR Simulation Performance
Tempo Despendido pela Simulação
300
250
(segundos)
Tempo Despendido
Time
(seconds)
Time of the Human Head SAR Simulation
SAR na Cabeça Humana
(dimensions variations)
200x200
200
150
100
400x400
600x600
800x800
1000x100
50
0
1
2
3
4
5
Quantidade de Processos
number
of process
Efficiency
Percentual de Eficiência
Eficiência
SAR
na Cabeça
HumanaSimulation
EfficiencydaofSimulação
the Human
Head
SAR Parallel
1,2
1
1 Processo
0,8
2 Processos
0,6
3 Processos
0,4
4 Processos
0,2
5 Processos
0
200x200
400x400
600x600
800x800
1000x1000
Dimensões dos
Domínios
Computacionais
Dimensions
of the
computational
domain
Carlos Henrique
43
Cluster Application: SAR Simulation varying the time steps
Eficiência
efficiency
Eficiência
SAR Simulation
Efficiencyda
in Simulação
the Human Head
SAR na Cabeça
Humana (400x400)
(400x400)
1,2
1
0,8
0,6
0,4
0,2
0
1 Processo
1 process
2 Processos
2 processes
3 Processos
3 processes
4 Processos
4 processes
5 Processos
5 processes
1000
2000
3000
4000
5000
Quantidade
de Passos
Tempo
Number
of timede
steps
Carlos Henrique
44
Cluster Application: SAR Simulation
Slice 1
Slice 2
Slice 5
Slice 4
Slice 3
Carlos Henrique
45
Cluster Application: Metamaterial simulations
Tempo Despendido
Time(seconds)
(segundos)
Tempo das Simulações
Metamaterial
2D
Metamaterial
Simulation
Time
Time (seconds)
2000
1500
1000
3000x323
AEL 1
1552
AEL2
814
AEL3
596
AEL10
468
AEL8
457
500
0
1
2
3
4
5
number
ofdeprocess
Quantidade
Processos
Eficiência do Processamento Paralelo
Metamaterial Simulation Time
1,2
efficiency
Eficiência
1
0,8
3000x323
0,6
0,4
Efficiency =
0,2
0
1
2
3
4
5
Tsequential
Tparallel
Quantidade de Processos
number
of process
Carlos Henrique
46
CePOF Cluster
0
1
Public Key - DSA
Internet
Router
Gateway
Firewall
Carlos Henrique
Intranet
47
CePOF Cluster: New Cluster Paradigm
•
15 nodes:
•
2 AMD Opteron 246 processors
•
4 GB of memory RAM
•
73 GB of HD SCSI
•
Switch 3COM 2824 (Gigabit Network)
•
Patch Cord Kat. 6
•
No break APC SURT 10000, GREE Air-Conditioner with 42.000 Btus
S.O. Virtual S.O. Virtual
Hw. Virtual Hw. Virtual
Virtual Machine
Windows 2003 Server
A2210 (AMD Opteron)
Carlos Henrique
48
Conclusions
• Computer Cluster allows high performance computing (HPC) with low costs;
• Free software and microcomputer reused (Beowulf cluster);
• Complex structure to configure, to use and to keep;
• Choose better Operational System distribution in order to improve the
computational performance;
• Making possible to simulate and analyze new electromagnetic structures and
many other applications.
Carlos Henrique
49
References
•
A. S. Tanenbaum, “Redes de Computadores 3ª. Edição”, Editora Campus, 1996.
•
C. A. F. Oliveira, “Cluster Beowulf- uma solução de baixo custo”, Revista CienteFico, Ano IV, V. I, Salvador, 2004.
•
C. H. S. Santos, H. E. Hernández-Figueroa, “Método FDTD 2D Paralelo para Simulações de Estruturas Metamateriais”,
12º Simpósio Brasileiro de Microondas e Optoeletrônica (SBMO) e 7º Congresso Brasileiro de Eletromagnetismo
(CBMAG), Belo Horizonte, 2006.
•
C. H. S. Santos, L. A. Ambrosio, H. E. Hernández Figueroa, “The Benefits of Heterogeneous Beowulf Cluster on the Human
Head SAR Simulation”, International Microwave & Optoelectronics Conference (IMOC), IEEE-SBMO, Brasília (2005)
•
C. H. S. Santos, “Computação Paralela Aplicada a Problemas Eletromagnéticos Utilizando o Método FDTD”, dissertação
de mestrado, Faculdade de Engenharia Elétrica e de Computação (FEEC), Universidade Estadual de Campinas
(Unicamp), Campinas, 2005.
•
J. J. Dongarra, et al, “Numerical Linear Algebra for High-Performance Computers”, Society for Industrial and Applied
Mathematics (SIAM), Philadelphia, 1998.
•
M. A. F. Batista, L. N. F. Guimarães, “Algoritmos Genéticos em Ambientes Paralelos”, V Worcap INPE, São José dos Campos,
2005.
•
M. Pitanga. Construindo Supercomputadores em Linux Segunda Edição, Editora Brasport, São Paulo, 2004.
•
N. S. Simões, S. F. Souza, L. Muniz, F. Fardim Junior, A. F. Souza, Reis N. C. Junior, A. M. P. Valli, L. Catabriga,
“Instalação e Configuração de Clusters de Estações de Trabalho: Experiência do Laboratório de Computação de Alto
Desempenho do Departamento de Informática da UFES”, IV Workshop em Sistemas Computacionais de Alto Desempenho WSCAD'2003, São Paulo, 2003.
•
P. S. Pacheco, “Parallel Programming with MPI”, Morgan Kaufmann Publishers, San Francisco, 1997.
•
P. Veríssimo, L. Rodrigues, “Distributed Systems for System Architects”, Kluwer Academic Publishers, Massachusetts, 2001.
•
“Introdução
ao
MPI”,
Centro
Nacional
de
Processamento
de
http://www.cenapad.unicamp.br/servicos/treinamentos/mpi.shtml (acessado em: 18/04/2007).
•
S. Humphries, “Field Solutions on Computer”, CRC Press LLC, Florida, 2000.
•
“Curso de MPI”, Centro Nacional de Processamento de Alto Desempenho-NE.
•
http://www.Beowulf.org/overview/history.html
(acessado em: 18/04/2007).
•
http://www.Beowulf.org/overview/faq.html#17
(acessado em: 18/04/2007).
•
H. Zima, B. Chapman, Supercompilers for Parallel and Vector Computers, ACM Press, New York, 1991.
Carlos Henrique
Alto
Desempenho-SP,
50
Thank you!
Carlos Henrique da Silva Santos
[email protected]
Leonardo André Ambrosio
[email protected]
Hugo Enrique Hernández Figueroa
[email protected]
Carlos Henrique
51
Network: Data Communication
Client machine
Client
process
(1) Connect request
Server machine
(2) ACK
(3) Request data
(4) Reply
System
calls
Operating
system
Protocol
Kernel
Drivers
stack
Carlos Henrique
Server
process
(5) Disconnect
(6) Disconnect
Kernel
Protocol
Drivers
stack
52

How to build a Beowulf Cluster Applied to Computational

Transcrição

Documentos relacionados

UCL Booking list before Final

Logarithmic transformation in the path analysis in experiments with

International Results - International Penguin Class Dinghy Association

Programme | 2014 September, 18th to 20th

Ride5 Creative Fashion Look Book

DEBIT AUTHORIZATION_Balloon - Terra Sol Turismo e Eventos

Corporate Taxpayer`s ID (CNPJ) 33.009.911/0001

Ian Astbury | Great new interview!

henrique tyszler - paihenriquedopc.com.br

Subject Program

mainstreaming examples - European Cluster Collaboration Platform

Discussion Paper_ Promoting Service Innovation through Cluster

Do German physicians want electronic health services? A

ii hedge fund data - Universität St.Gallen