slides - Terena

Implementation of a
new Optical Platform in
X-WiN
Peter Kaufmann/DFN
TERENA Network Architects Workshop
22. November 2012
Agenda
• Reasons & Requirements
• Results
• Status of Implementation
Page 2
Reasons & Requirements (1)
Why change?
• Capacity: Less than 50% of our lambdas are in use!
• Upgrade towards 40/100G only with new h/w
• Current equipment is approaching end of life (from 2006)
(investments into old h/w not usefull)
Most important
• Operational inflexibilities very annoying
– as lambdas has
to be changed
– as more native lambda services are provided/added
– as transponders are dedicated/configured for each connection
Page 3
Reasons & Requirements (2)
Add flexibility
• Transparent lambdas between any core nodes (no OEO)
• Protected lambdas
• Easy and quick channel setup
– rapid service delivery
–
flexible use of transponders
– ability to reroute lambdas
easily in case of
serious link problems
• Optical switching facilities at fiber junctions
• Sub-Lambda switching facilities (OTN) (due to 10/100)
Page 4
Reasons & Requirements (3)
But also: Increase transport capacity
• compensate traffic growth
• native 100G/channel capability for our most demanding
users
Page 5
Reasons & Requirements (4)
New optical transmission system
•
•
•
•
About 80 lambdas
Support for native 10G & 100G services
Transparent lambdas between any core nodes (~1000km reach)
ROADM: directionless & colorless (not fully contentionless, no flexgrid),
integrated OTN-Switching
•
Migration philosophy
– On top of current fiber infrastructure (only
few add. fibers)
– 100G transponders where needed, when needed
– Smooth migration, no operational „shut down“
Seite 6
Results
• Call for tender (non open, 2 stages): November 2011
– After stage one, reduction to „some serious“ participants
• Acceptance of tender: 10. May 2012
• Winner
–
–
–
–
ECI Telecom, http://www.ecitele.com
Israelien company, founded 1961
about 2500 employes
New player in „NREN environment“
• Contract & order for migration: 14. May 2012
• Product family „Apollo“
– Optimized Multilayer Transport (OMLT)
– integrated DWDM-/Switching-equipment
– DFN: OPT9624 at all core nodes (plus OPT9608 for some clients)
Seite 7
Results: ECI OPT96xx „Apollo“
 24/48 universal I/O slots
OPT9624/48 for Metro
Core and Core/Regional/LH
 Tbit/s Universal Switch:
ODU X-connect/Packet Switching
 Flexible configuration:
Photonics, ODU-XC, Packet switching
 8 universal I/O slots
OPT9608 for Metro Edge
 Flexible configuration: standalone WDM,
100G MPLS switching capacity
OPT9604 for Metro Edge
OPT9603 for Metro access
and In-Line amplifier
Artemis
 4 universal I/O slots
 50G MPLS switching capacity
 OPT9603 - 2U height with 3 universal slots
 Artemis: passive cages
Quelle: ECI Telecom
Seite 8
Results: Fiber set-up
• High Capacity
– 88 Lambdas per Fiber
– Bandwidth up to 100 Gbit/s per Lambda
• Flexible Routing of Connections
– Reconfigurable Optical Add-Drop Multiplexer (ROADM)
– Colourless/Directionless Add-Drop
– Tunable Transponder
• Switching Functions
– Optional: 1Tbit/s Switching-Fabric per Node
– Currently: Usage as ODU Cross-Connect
– Future: In addition usage as MPLS- and Ethernet-Switch
Seite 9
Capacities in Generations of WiN
Bandwidth in Core of WiN (Wissenschaftsnetzes):
X-WiN 2012: 8.800 Gbit/s
X-WiN 2006:
400 Gbit/s
G-WiN 2000:
10 Gbit/s
B-WiN 1995:
0,622 Gbit/s
Seite 10
Results: Fiber set-up
Amplifier
• Raman amplifiers compensate excessive attenuation on
long spans
• New: Mixture of Raman, EDFA
• Requires additional maintenance procedures
Redundancy
• Customer connection: Redundant fiber with two Core Nodes
(nearly all customers, 77% had not even 1 second interruption in 2011)
• Further extension of redundant fiber connectivity
Seite 11
Results: ROADM
• High Capacity
– 88 Lambdas per Fiber
– Bandwidth up to 100 Gbit/s per Lambda
• Flexible Routing of Connections
– Reconfigurable Optical Add-Drop Multiplexer (ROADM)
– Colourless/Directionless Add-Drop
– Tunable Transponder
• Switching Functions
– Optional: 1Tbit/s Switching-Fabric per Node
– Currently: Usage as ODU Cross-Connect
– Future: In addition usage as MPLS- and Ethernet-Switch
Seite 12
Results: ROADM
ROADM functionality
•
•
Colourless: Flexible A/D on any available Lambda
Directionless: Flexible mapping on any required Direction
(But still uni-directional use of each part of fiber pair)
•
Contentionless: Blocking-free switching of same colours, requires add. WSS
•
(Not that important for DFN, partially possible, not 100%)
Flexgrid/Super Channels: Not required for DFN use cases during life time of gear
•
•
•
Each ROADM configuered with at least: 4 degrees + 2 A/D blocks,
extension to 9 degrees possible
OTN-Switch complements sub-lambda switching done in routers
Restoration by ROADM: Switch time 1sec
Seite 13
ROADM: Optical Architecture
Source: ECI Telecom
Page 14
ROADM: L1 Service Cards
Client:
Multi service 10G
Muxponder or AoC
Multi service 10G
double
Transponder
Multi service 40G
RZ-DQPSK
Muxponder
AoC10
SFP
16 x SFPs
STM-1/4/16, OC3/12/48
FC1/2/4, GbE,
SDI, HD-SDI, DVB-ASI
OTU1
Client:
TR10_4
CMB40
SFP
OTN
Mapper
XFP
2 lines
OTU-2/2e
SFP
SFP
XFP
SFP
XFP
10G LAN,
STM64/OC192,
FC8/10,
OTU-2/2e
XFP
Client:
XFP
OTN
Mapper
XFP
2 lines
OTU-2/2e
10G LAN,
STM64/OC192,
FC8/10,
OTU-2/2e
XFP
XFP
OTN
Mapper
QPSK
XFP
Line
OTU-3e
XFP
OTN
Mapper
OTU3 Regenerator
REG40
QPSK
QPSK
Line
OTU-3e
Transponder/
Muxponder 100G
Line
OTU-3e
OTN
Mapper
TR100
QPSK
Line
100GbE
QPSK
Line
OTU4
Source: ECI Telecom
Page 15
ECI: ROADM/Service-Cards
North
West
East
Local A/D
Without Fabric: L1 Service Cards
AoC10: 16*SFP -> 2*XFP-OTU2
TR10_4: 2*XFP -> 2*XFP-OTU2/2e
TR100: 1*CFP -> 1*CFP-OTU4 (later)
With Fabric: L1 Fabric Cards
Fabric
FIO10_5: 5*XFP -> Fabric
FIO100: 1*CFP -> Fabric (later)
FIO10_5: 5*XFP -> Fabric (later)
FIOMR_16: 16*SFP -> Fabric
Client
page 16
Results: Switching
• High Capacity
– 88 Lambdas per Fiber
– Bandwidth up to 100 Gbit/s per Lambda
• Flexible Routing of Connections
– Reconfigurable Optical Add-Drop Multiplexer (ROADM)
– Colourless/Directionless Add-Drop
– Tunable Transponder
• Switching Functions
– Optional: 1Tbit/s Switching-Fabric per Node
– Currently: Usage as ODU Cross-Connect
– Future: In addition usage as MPLS- and Ethernet-Switch
Seite 17
Optical Transport Network (OTN)
Source: Alcatel-Lucent
Seite 18
OTN-Hierarchy
Source: Alcatel-Lucent
page 19
Switching: OTN-BB of X-WiN
KIE
• Operation of 1Tbit/s-Fabric (blue-red)
DKR
DES
AWI
– Initially at 14 core nodes
– Optimized with current network
structure and latency
– If needed: extensible
GRE
ROS
HAM
EWE
BRE
ENS
HAN
BIE
MUE
DUI
• OTN-Backbone (yellow)
FZJ
– Start: 2-3 OTU2-Connections per link
– Future: Extension towards OTU3/4
PAD
DOR
WUP
BIR
AAC
BON
– All interfaces use OTH-framing
– Single or redundant OTU2e-connection to
next one/two core nodes with fabric
BRA
KAS
MAR
FRA
FZK
STB
KEH
BAS
HUB
ADH
LEI
DRE
CHE
ILM
WUE
SAA
SLU
ZEU
POT
ZIB
MAG
JEN
GIE
KAI
FFO
GOE
GSI
• Access from User/Simple Core
Node to Fabric
PEP
TUB
BAY
ERL
ESF
REG
HEI
STU
FHM
GAR
Seite 20
Switching: L1 Fabric Cards
Low rate client
interface
Multi service
10G client line
Interface
40G RZ-DQPSK
line card
100G PM-QPSK
line card
FIOMR_16
FIO10_5
16 x SFPs
STM-1/4/16, OC3/12/48
FC1/2/4
GBE,
SDI,HD-SDI, DVB-ASI
OTU1
SFP
SFP
1Tbps
ODU-XC
SFP
SFP
SFP
XFP
5 x XFPs
10G LAN,
STM64/OC192,
FC8/10,
OTU-2/2e
OTN
Mapper
XFP
OTN
Mapper
1Tbps
ODU-XC
XFP
XFP
XFP
OTN
Mapper
FIO40
1Tbps
ODU-XC
QPSK
Line
OTU-3e
OTN
Mapper
FIO100
QPSK
Line
OTU-4
1Tbps
ODU-XC
Source: ECI Telecom
Page 21
ECI: ROADM/Fabric-Cards
North
West
East
Local A/D
Without Fabric: L1 Service Cards
AoC10: 16*SFP -> 2*XFP-OTU2
TR10_4: 2*XFP -> 2*XFP-OTU2/2e
TR100: 1*CFP -> 1*CFP-OTU4 (later)
With Fabric: L1 Fabric Cards
Fabric
FIO10_5: 5*XFP -> Fabric
FIO100: 1*CFP -> Fabric (later)
FIOMR_16: 16*SFP -> Fabric
FIO10_5: 5*XFP -> Fabric (later)
Client
page 22
Universal Switch Fabric
 The fabric can support ODU-XC,
MPLS switch or any mix of both
(depending on the service cards
installed in the system).
 Scalable to 4Tbit and future
16Tbit switching capacity (in
multi-shelf Architecture)
ODU-XC
(ODU-0-4, Flex)
10GbE
/ STM1-64 /
FC10 / OTU-2
10G
OTN
OTN
OTN
OTN
OTN
OTN
Packet
Packet
Packet
Packet
Packet
Packet
STM1 / 4 / 16 /
FC1 / 2 / 4 / GbE
40G
100GbE
100G
 ODUk and ODU-Flex XC
 Transparency of timing and OH;
Each ODUk container carries its
own timing
 Each line card is configured to
work towards the fabric in either
OTN mode or data mode
100GbE
100G
10GbE
10G
1GbE
100G
Packet
Switching
Source: ECI Telecom
Seite 23
Status: Migration Work
• Core Nodes (54)
– about 100 new chassis (power, etc.)
• Intermediate Line Amplifier: 45 Locations
• Fiber links (85)
– Total length about 10.500 km
– 33 parallel lines between central Core Nodes
• Lambda connections (167)
– Currently: Gigabit-Ethernet and 10-Gigabit-Ethernet
– With max. 18 hops and 2140 km fiber length
Page 24
Status of Implementation
Planning ECIComponents
Provision and Operation ECIComponents
Planning parallel Fiber
Infrastruktur
Provision Parallel Fiber Infrastruktur
Planning
LocationInfrastructure
Implementation Location
Infrastructure
Training
of DFNEmployes
Preparation of Network Monitoring and
Adaption of Information Systems
05
06
07
08
09
10
11
12
Seite 25
Conclusion: Seen from Customer
• Increase of Bandwidth
– 100Gbit/s-Connections for VPN-service and DFNInternet (IP)
• Reduction of Provisioning Time for (new) Services
– Equipment is property of DFN: Better reserve of often required
components
– Flexibility: Components may used again for other connections
• New Switching Service
– Now: OTN, Mux of several 1Gb VPN over one 10Gb access
– Later: MPLS and Ethernet
• Reduction of Cost
– Components for 1Gbit/s- and 10Gbit/s-Connections much cheaper
Page 26
Questions ...?
?
?
?
Seite 27
ROADM – classical
•
coloured
– For each „colour“ a specific port
per Add/Drop-Block
•
Directed
– Exactly one outgoing direction erreichbar
per Add/Drop-Block
T
/
R
T
/
R
Add/
Drop
North
DWDMKnoten
Add/
Drop
West
Add/
Drop
East
T
/
R
T
/
R
T
/
R
X
T
/
R
Seite 28
ROADM – „modern“
•
Colourless
– each „colour“ at any Add/Drop-Port
•
Directionless
– each Ausgangsrichtung erreichbar at
any Add/Drop-Port
DWDMKnoten
Add/Drop
T
/
R
T
/
R
T
/
R
T
/
R
T
/
R
Seite 29
Gigabit AoC / Client-Card
2xXFP OTU2 colored uplinks
16xSFP colored uplink
Source: ECI Telecom
Seite 30
10G Transponder / Fabric Card
2xXFP FC8/10,STM-64/OC192/10GBE/OTU2/2e B&W or
colored I/Fs
2xXFP OTU2/2e/2f colored uplinks
5xXFP FC8,STM-64/OC-192/10GBE/OTU2/2e B&W or colored I/Fs
Source: ECI Telecom Seite 31
Variants of „Apollo“
Pure WDM Application
 24 universal slots
 Photonics modules
 Service cards
 L1 service cards
Metro Core / Regional / LH OMLT
Photonic layer
 4 slots for 1Tbps
TR100
Mux100
TR40
Mux40
TR10
AoC10
modules
 Service cards
Photonic layer
TR100
Mux100
TR40
Mux40
TR10
AoC10
 L1 service cards
 L2/3 Data cards
1GbE
L2/3
100G
Data
Fabric
Line
100G
10G
Line/Client
Client
100G
 L1 service cards
1GbE
L2/3
 L2/3 Data cards
10GbE
L2/3
Line
40G
ODU-XC
10G
Line/Client
Packet
Switching
100GbE
L2/3
CESR Application
fabric cards
 22 universal slots
 Photonics
Low
Rate
 20 universal slots
 Photonics modules
 Service cards
Metro OMLT Application
 2 slots for 100G
Photonic layer
universal fabric
cards
 4 slots for 1Tbps
universal fabric
cards
 20 universal slots
 Service cards
Photonic layer
1GbE
L2/3
10GbE
L2/3
100GbE
L2/3
Packet
Switching
 L2/3 Data cards
1GbE
L2/3
10GbE
L2/3
100GbE
L2/3
10GbE
L2/3
Source: ECI Telecom
Seite 32
Service Restoration
3rd Restoration
Path
Main Path
E
A
F
1st Restoration
Path
C
G
2nd
Restoration
Path
H
A
S
P
L
I
T
Restoration by ROADM. Switch time 1sec R
O
A
D
M
C
Seite 33
Deutsches
Forschungsnetz
Bereitstellung Glasfaserstrecken
KIE
Glasfaserpaar
,(Bestand,
nicht parallel)
DKR
DES
AWI
Glasfaserpaar
(vorhanden)
GRE
ROS
HAM
EWE
Glasfaserpaar
(parallel, beauftragt)
ENS
Glasfaserpaar
(parallel, Übergabe
erfolgt)
HAN
MUE
DUI
Glasfaserpaar
(parallel, Fertigstellung
bis Ende Oktober)
FZJ
AAC
BON
BIE
BOC
DOR
WUP
BIR
GIE
PAD
KAS
MAR
FZK
STB
KEH
BAS
HUB
ADH
LEI
DRE
JEN
CHE
ILM
FRA
KAI
BRA
POT
ZIB
MAG
SLU
GOE
WUE
GSI
SAA
FFO
TUB
ZEU
BRE
HEI
BAY
ERL
REG
STU
FHM
GAR
Seite 35
Stand Installation der Technik
KIE
Glasfaserpaar
,(Bestand,
nicht parallel)
Glasfaserpaar
(vorhanden)
ROS
FFO
TUB
ZEU
BRE
ENS
HAN
Kernnetzknoten
(bereits aufgebaut)
MUE
DUI
Kernnetzknoten
(geplant KW42)
Kernnetzknoten
(geplant KW44)
HAM
EWE
Glasfaserpaar
(parallel, beauftragt)
Kernnetzknoten
(geplant KW43)
GRE
DKR
DES
AWI
FZJ
AAC
BON
BIE
BOC
DOR
WUP
BIR
GIE
PAD
KAS
MAR
KAI
FZK
STB
KEH
BAS
LEI
DRE
JEN
CHE
ILM
WUE
GSI
SAA
HUB
ADH
GOE
FRA
Kernnetzknoten
(geplant KW45)
BRA
POT
ZIB
MAG
SLU
HEI
BAY
ERL
REG
STU
FHM
GAR
Seite 36
Beteiligte Einrichtungen
•
DFN Geschäftsstellen in Berlin und Stuttgart
•
ECI Telecom
– beschäftigt zusätzlich Montagefirma für Aufbau DWDM-Technik (7 Teams)
– zusätzlich 6 eigene Teams für die Inbetriebnahme
•
Glasfaserprovider
– stellen parallele Faserstrecken bereit und Stellflächen/DC-Versorgung an
ILA-Standorten
– insgesamt 10 Firmen
•
Dimension Data
– Installationen an Kernnetzknoten (koordiniert zusätzlich Montagefirmen für
Schrankaufbau und Installation von Vorverkabelungen)
– Ersatzteillogistik (Zentrallager plus 10 regionale Depots)
•
Gastgebende Einrichtungen an Kernnetzknoten
– zusätzlich Elektriker für die Installation von Stromversorgungen
Seite 37
Installationen an Kernnetzknoten
•
48V/DC Netzteile
– je 1-2 Chassis für USV/NS
– Kapazität je Gerät 4kW (zzgl. 2kW Spare)
•
Fibre-Spooler/Cable Guides
– „saubere“ Führung von Glasfaserkabeln
•
Passivgehäuse „Artemis“
– Aufnahme von passiven Modulen
(bspw. optische Splitter)
•
Apollo 9624
– 1-4 Geräte je Standort
– Aufnahme der aktiven Komponenten
(Verstärker, ROADM, Line-/Fabric-Karten)
Seite 38
Installation am Standort JEN
•
KNK Universität Jena (JEN)
– drei Kernnetzfasern
– zehn 10G-Verbindungen
– drei ECI OPT9624 Chassis
•
Stellflächen
– Aufbau in nur einem Datenschrank,
Bestückung von Vorder- und Rückseite
– bisher wurden 3 Stellflächen genutzt
•
Spannungsversorgung
– komplett redundant bzgl. Equipment und
Versorgungszweig
– kann zukünftig überwacht werden
Seite 39
Installation am Standort JEN
•
Separates Chassis für ODU-XC
– drei 5-Port 10G-Karten
– je 2-3 OTU2 Trunks zu anderen XCs
Seite 40