Vortrag

Transcrição

Vortrag

TechDemo Backup Deepdive
15:15 - 16:45
Deduplication of
Big Data
&
TSM Storage:
wohin mit Big Data
[email protected]
Solutions Architect Data Protection
© 2014 NetApp. Alle Rechte vorbehalten.
1
Non Disclosure Agreement
This presentation/document is the exclusive property of
NetApp, confidential and does neither constitute a part
of nor can it be incorporated into any contractual
agreement with NetApp or its subsidiaries or affiliates.
Roadmap information contained in this
presentation/document is for informational purposes
only and is intended solely to assist you in planning for
the implementation and upgrade of the product features
described.
It is not a commitment to deliver any material, code, or
functionality, and should not be relied upon in making
purchasing decision.
The development, release, and timing of any features or
functionality described in this presentation/document
remains at the sole discretion of NetApp.
13th November 2014 (collected by Dieter Unterseher, no guarantee)
2
NetApp Deutschland
Die Fakten
Marktführer
28,0%* Marktanteil im Bereich
Open Systems Storage in Deutschland
Platz 1 beim Gesamtmarktanteil*
>25.000 installierte
Systeme
>4.000 Kunden
~600 Mitarbeiter in 8 Niederlassungen
*Quelle: IDC Disk Storage Systems Tracker März 2014,
FAS Marktanteil Deutschland (w/o ESCON/DAS), Umsatz 4-Quartale (R4Q)
3
Chapters
Deduplication Technology for
Data Protection
Storage for DPR for Big Data
DP Methods / Trends for Big Data
4
Chapter “Deduplication Technology
for Data Protection”
Dedupe Fundamentals
Dedupe Targets for DPR
NetApp SteelStore
ONTAP Deduplication & Compression
5
Deduplication
Technology for
Data Protection,
Fundamentals
“deduplizierbare” Datenbestände
7
How do backup apps write data to tape?
General/typical data serialization…
File
Header
File
Footer / Pad
Tape
Backup
#1
Image
Heade
r
Tarball
8
What if we add slides to Pres_1.ppt?
Tape
Backup
#2
Image
Heade
r
Byte Shift
9
What if we deleted Doc_1.doc?
Tape
Backup
#3
Image
Heade
r
Byte Shift
10
What if we created Txt_1.doc?
Tape
Backup
#4
Image
Heade
r
11
How backup apps write data to tape
And also to disk (in many cases!)…
Data is packaged serially into proprietary “tarball” formats,
which are then written to tape (or disk)
Creating new files/data in an environment usually results in
“insertions” into backup data sets, and byte shifting to the
right
Removal of files/data from an environment usually results in
deletions in backup data sets, and byte shifting to the left
If data alignment happens at all, padding to 512-byte
boundaries is most common in tape formats
– Examples: Symantec NetBackup, UNIX tar(1)
12
Alignment Sensitivity in Dedupe
Fixed block, fixed offset dedupe…
Example: Block boundaries are every 4 shapes
≠
Nothing matches anymore!
Insert One
Shape
13
Alignment Agnostic Dedupe
Variable block, variable offset dedupe…
Example: Block boundaries occur when there are
two triangles in sequence
Block boundaries move with the data
Insert One
Shapeof blocks
Lots
still match!
14
GBC Dedupe Operations
Anchor Generation
– Used to do initial markup of opaque backup
streams
– Utilizes a fast rolling hash function to mark block
boundaries in a data dependent manner
– Anchors are persistent and aggressively cached
for high speed lookup
Grow by Compare (GBC)
– Used to identify duplicate sequences of data in
backup streams
– Utilizes high speed sequential byte compare
– Duplicate extent lengths are not size-constrained
by anchor boundaries
15
Getting started: Anchor Generation
!
A
!
B
!
C
!
D
!
E
Anchors are generated using a rolling hash algorithm
Hash is computed on a small window of data that slides through
the input stream byte-by-byte
A hash becomes an anchor when it is observed to have “special
properties”
Algorithm typically generates ~3 anchors per 64K of data
–
Anchor generation rate can be controlled/tuned
Anchor Generation can run either inline or post-processed
16
The dedupe Process
Unique data is added to common object store…
A
B
C
D
E
Common Object Store
17
The dedupe Process
The next backup arrives…
A
B
C
New data has appeared in the backup
Common Object Store
A
B
C
D
E
18
Phase 2: Grow by Compare
A
B
C
REF A
Common Object Store
A
B
13th November
C
D
2014 (collected by Dieter Unterseher, no guarantee)
E
19
Anchor Generation + More GBC
C
REF A
REF B
Common Object Store
A
B
13th November
C
D
E
20
New Data for Common Object Store
REF A
REF C
REF B
Common Object Store
A
B
13th November
C
D
E
21
The dedupe Process
And so it continues…
REF A
REF C
REF B
Common Object Store
A
B
13th November
C
D
E
22
Skips
Keeping Grow-by-Compare efficient…
File headers within a backup stream may contain:
– Last Access Times
Modified when a file has been accessed (but not changed) since the
last backup was done
– Other file metadata that may change between backups
Filename
Owner, Group Owner(s)
Security Permissions
…
– Backup application metadata that may change between backups
Timestamps
Sequence numbers
Checksum fields
…
Skips make grow-by-compare extremely efficient in locating and
eradicating duplicate segments
23
Skips
Skips efficiently accommodate small variations between
otherwise duplicate data segments
A
B
Skip Repository
Common Store
Segment
A
B
24
Skips
Skips efficiently accommodate small variations between
otherwise duplicate data segments
A
B
Skip Repository
Common Store
Segment
A
B
25
Skips
Virtual tapes reference Common Object Store
And sequentially stored skips
REF A
REF B
REF C
D
(etc)
26
DP target
Deduplication
one Option for
DP Generation 2
(classical D2D backup)
27
Data Protection Deduplication
Findet die komplette Dedupe-Reduzierung
beim DP-Dedupe-Target statt, spricht man von
„target depuplication“ (auch „backend
Dedupe“ / „server side Dedupe“ genannt):
VTL / NAS / OST-Appliances mit DedupeFunktionalität
DP-Serversoftware mit DedupeFunktionalität (z.B. TSM ab 6.1)
Findet die Dedupe-Reduzierung in Absprache
zwischen dem DP-Client und dem DedupeTarget statt, spricht man von „client side
Deduplication“
28
DP-target Verdichtungsmethoden
Deduplizierung und Kompression
1. Dublettenerkennung
2. Komprimierung
Zum Verdichten von klassischen Backup-Datenströmen muss vor obiger
Reihenfolge zunächst ein Chunking per Variable Byte Range Logik nach
Dateninhalten (unabhängig von physischen Blockgrenzen) erfolgen.
Die Dedupe-Ratio ergibt sich durch Multiplikation der Dubletten& Komprimierungs-Faktoren (z.B.: 5:1 Dubletten-Erkennung und 2:1
Komprimierung ergeben
(5x2=)
10:1).
29
Was beeinflußt die DP Deduplication-Ratio?
(am Beispiel tägliches Incremental- & wöchentliches Fullbackup)
Time
Growth
So
Mo
Di
Mi
Do
Fr
Sa
So
1)
2)
3)
Daten-Dubletten im ersten Full-Backupzyklus (stark datenabhängig, z.B. 1,3:1, nicht dargestellt)
Die File-Änderungsrate zwischen Filelevel Incremental-Backups (z.B. 1,3:1)
Die über 1 hinaus gehende Anzahl gehaltener Fullbackups (abzüglich Change-Rate zwischen den
Fulls) hat den größten Einfluss auf die Dedupe-Ratio (z.B. 3:1 oder 10:1).
4)
Die Komprimierbarkeit der Backups (z.B. ~3:1 bei Oracle, oder ~1,5:1 bei Officefiles)
Das Produkt der Faktoren dürfte (bei 20 Fulls mit 5% Change-Rate & 2:1 Komprimierbarkeit) grob 20:1 DedupeRatio ergeben (die Ratio verändert sich nicht linear zu der Anzahl Fullbackups).
30
Nominal hohe Dedupe-Ratios blenden –
die Optimierung der DP-Prozesse sind das Ziel
Dedupe-Ratios über 5:1 bringen nur wenig zusätzlichen Platzgewinn.
Deduplication selbst löst keine Backup-/Restore-Probleme
(es vermindert nur den Diskbedarf)
31
DP Compression / Deduplication Design:
3 mögliche Verdichtungsoptionen denkbar
Compression - only
Post-ProcessDeduplication
Inline-Deduplication
Compression only ist zu bevorzugen:
wenn höchste Lesespeed gefordert ist (denn für Tapekopien / Restores ist dies nach Deduplication
aufgrund stark erhöhter Disk Random Reads nur noch beschränkt möglich)
wenn wenig Dubletten im Datenstrom sind
practical
wenn man Dedupe-Zusatzrisiken scheut
hints
Deduplication ist zu bevorzugen, wenn höchste Diskeinsparung gefordert ist & nenneswerte
Dubletten vorhanden sind & die Restore / Lesespeed zweitranging ist:
Postprocess Dedupe erlaubt (wo geboten) eine schnelle, risikoärmere Tapekopie vor dem Deduplication
(aber Postprocess Dedupe sollte hier immer binnen 24h fertig werden).
Inline-Deduplication skaliert bei Rund-um-die-Uhr Backups besser und spart die Diskkapazität der
Zwischenspeicherung.
32
Was ist das “Hash Key Collision Risiko”? practical
hints
Prüft eine Dedupe Implementierung bei gleichem Hash Key, ob der
(oft ~200-fach) längere Datenbereich wirklich den gleichen Inhalt hat?
Die meisten Dedupe-Implementierungen arbeiten Hash-basiert und prüfen dies
nicht.
Das erhöht die Speicherungs-Performance erheblich (viel weniger IOs) und
erlaubt den Einsatz großer und Kapazitäts-Disks –
Aber dies ist mit dem kleinen Restrisiko „silent data loss“ verbunden.
Sollte dies auftreten, droht dauerhafter Datenverlust:
über alle Backup-Generationen hinweg und
für daraus erstellte Non-Dedupe Kopien (z.B. Backup-Kopie auf Tape)
Welche Dedupe-Implementierungen arbeiten mit diesem Restrisiko?
NetApp‘s Implementierungen arbeiten ohne dieses Restrisiko (Dedupe nur nach
erfolgreichem Byte2Byte Datenbereichsvergleich).
Alle Dataprotection-Clients mit client side Dedupe Funktion arbeiten mit dem Restrisiko.
~ 85% der Dedupe Appliances am Markt verwendeten arbeiten mit dem Restrisiko:
Welche? Siehe unter: http://www.backupcentral.com/wiki/index.php/Disk_Targets,_currently_shipping
Alle, welche dort in Spalte “Dedupe Method” den Inhalt nur “hash” stehen haben, arbeiten mit dem Hash Key Collision Risiko.
33
Einsatz von Deduped Replication?
practical
hints
Die meisten Dedupe-Appliances bieten Deduped-Replication:
Dies kann helfen, eine DR-Kopie der Backups per IP in eine
andere Niederlassung zu bewegen.
aber ist mit hohen Gesamtkosten verbunden
und erzeugt einiges an Metadaten-IP-Traffic.
34
Deduplication von DP-Medien hat auch Nachteile
(gegenüber Compression-only)
Die Lesespeed von deduplizierten Backups wird geringer
(stört ein performantes Erstellen von Tape-Kopien)
VBR Deduplication ist komplex verpointert – birgt Risiken
Tägliche Housekeeping-Prozesse erzeugen viel Last
Es gibt Datentypen, für die Deduplication unvorteilhaft sind
35
OST Add On: Hybrid Deduplication („SPEED“)
(for NetBackup & BackupEXEC; Competitives call a similar Feature „Accent“ or „Boost“)
Normal Deduplication
Hybrid Deduplication
(all deduplication is performed on CS800)
(deduplication workload is distributed)
Media Server
Media Server
Media Server
Media Server
Media Server
Media Server
Media Server
Media Server
less bandwidth
used for
backups
all data sent
NAS
OST
VTL
H
O
ETERNUS CS800
1.) chunking &
hashkey-creation
OST
2.) check hashkey
against index
H
O
3.) for unique data only:
ETERNUS CS800
a.) compress
1.) chunking &
hashkey-creation
3a.) for unique data only:
compress
distribute workload
Gives increased
Aggregate Backup
Performance
2.) check hashkey
against index
3b.) for unique data only:
store
b.) store
same logic is also used for Client side Dedupe 
36
NetApp SteelStore
designed for Backup2Cloud
37
NetApp SteelStore, how does it work?
Dedupe & Compression offers optimized
Disk-Target usage & WAN-Traffic
SteelStore offers CIFS-Shares / NFSExports to write backup-Container-Files
or restore from them
The early & strong AES256
Encryption minimizes any
Security risks at local,
WAN/Internet-Connection
& at Cloud Provider
Filer2Server NDMP
Any
traditional
Backup Data
or any other DP Software like SnapProtect,
which is able to write backup-Files to
CIFS/NFS Shares
SteelStore does:
1. Deduplication
2. Compression
3. AES256 Encryption
4. Store on the local Disk-Cache
(for days to weeks)
5. bring new blocks async to a
cloud Device (for months)
Relatively cheap Cloud
Storage offerings as DP
target:
Amazon S3,
Microsoft Azure
etc
38
NetApp SteelStore, for what makes it sense
It is designed for Backup 2 cloud & easy to implement & to use
It has the potential to replace some physical Tapes, because:
– Used Cloud Storage is also very cheap
– Backups get very earlier into another DC (which results in much
lower Data loss at Disaster; no manual tape handling for off-siding
needed)
It has the potential to offer optimal small RoBo Backups (in case
NetApp offers the virtual SteelStore Appliance for production):
– Last Backups (~ 1-2 weeks) are kept automatically locally in the
Backup-Cache
– Only older backups (~ months) are on Cloud-Provider side (Single
File Restore)
39
NetApp SteelStore, Status
Since Nov 2014 on the NetApp Pricelist:
a Product- & Developer-buyout from Riverbed, which had
already with Version 3 ~165 Customers worldwide
NetApp is currently investing in SteelStore Skill to ramp up
# of models offered still under considerations (we start with the
Highend-Appliance; Virtual SteelStore Appliance are currently
usable for PoC)
In case you have some time to look for a Backup2cloud
solution, wait some weeks, until NetApp got enough
experience with the product.
40
ONTAP Deduplication
& Data Compression
(stark verkürzt behandelt;
das sehr etablierte Deduplication wird nur kurz behandelt)
41
NetApp Compression & Deduplication
dropped license keys for Compression & Dedupe with ONTAP 8.1
transparent to all Applications & Protocols (CIFS, NFS, SAN-LUNs)
works with SnapMirror*, SnapVault (clustered ONTAP), OSSV-targets,
MetroCluster*, SnapLock, vfiler Multistore &
NDMP Dump/SM2T*/SMTape* (*= keep the benefit over the line).
No reduced volume size limits for Dedupe & Compr. since ONTAP 8.1
Compression / Deduplication combinations per FlexVol for 7-Mode &
Clustered ONTAP
–
Inline Compression only
–
Inline Compression with postprocess Compression & Dedupe
–
Postprocess Compression & Dedupe (mainly for existing undensed data)
–
Postprocess Dedupe only
–
(Postprocesses are scheduled together; first Compr., followed by dedupe)
42
NetApp Deduplication
Flexibles Volume
ohne FAS Dedupe
(since ONTAP 7.2.5)
FAS Deduplication Prozeß
Metadaten
asynchron (Erstausführung, danach z.B. täglich)
Performanceimpact hier – Normalbetrieb zeigt fast
unverändertes Performanceverhalten
wirkt auf alle Blöcke des ActiveFileSystems. Der DiskEinspareffekt tritt somit zeitverzögert auf, sobald die
älteren Snapshots entfallen.
Flexibles Volume
nach FAS Dedupe
4KB Datenblöcke
Datenböcke gleichen Inhalts
werden freigegeben (nach
Byte2Byte Vergleich, also kein
„Hash Key Kollision“
Datenverlustrisiko)
etwas mehr
Metadaten
meist deutlich weniger
Datenblöcke
43
NetApp Deduplication
NetApp® deduplication frees duplicate WAFL® data blocks by
reference a single data block by multiple pointers
Support on R200 / FAS2xxx / FAS3xxx / FAS6xxx & V-Series
Inode
Inode
Indirect
Block
DATA1
Indirect
Block
DATA2
Indirect
Block
DATA3
Indirect
Block
Indirect
Block
DATA3
DATA3
44
ONTAP Data Compression
For all FAS and V-Series beginning with ONTAP 8.1
On FlexVol-Level in 64-Bit aggregates possible
Compressed Data Blocks are with 8.2 not cached in
the FlashCache- & FlashPool – Layers
Reduces Disk Storage needs & some physical IOs
(some experienced faster Speed with Compression)
Needs more Storage-Controller CPU- Cylces (some
saw under 8.2 in worst case up to 50% speed
reduction for some data with 100% occupied CPUs)
45
How Does Compression Work?
32k compression groups
188k
logical
Compression
Groups
abcdeabcdeaaabcdeaaabcdeabcdeabc
uvwyyabxzzabuyxzrcuvwyyxzrcabxzz
fghijklmopqfghijrstrstopqklmrstn
32K
32K
32K
32K
32K
abcdeabcdeaaabcdeaaabcdeabcd
24K
46
How Does Compression Work?
Immediate space savings
#!*~>abc
^@(%)/*n
with inline compression -&*+*-#@$abuy
post process is usable
Compressed
Data on Disk
for existing elder data
#!*~
abcdeabcdeaaabcdeaaabcde
-&*+
>abc
*#@$ abuy
^@(% )/*n
-&*+
*#@$ abuy
^@(% )/*n
#!*~
>abc
56k on disk
47
Typical Storage Savings & Settings
80%
70%
Compression Only
Deduplication and Compression
practical
hints
Deduplication Only
60%
50%
40%
30%
20%
10%
0%
Typical efficiency settings (for primary and secondary volumes) may be:
C+D
D
C
C
C
C
C+D
C
48
Compression and/or Deduplication What makes sense / best practice? (1)
practical
hints
Results may very, run SSET 3.0 to prove your data,
which of both or both or none makes sense per volume
With existing snapshots, prefer to activate / deactivate
it for future Writes only: This avoids additional Disk-Space
needs, until the Snapshot before the full volume dense /
undense expires.
A typically good schedule time for postprocess
Compression & Dedupe run is daily 23:05 : realize
reduction before the midnight snapshot, which is
typically held longer.
49
Chapter „Storage for DPR for Big Data”
Data ONTAP Storage-Systems
NetApp E-Series (an example for a
traditional Block-Storage)
Physical Tape?
50
Storage Systems for Big Data
Data ONTAP
Storage-Systems
51
NetApp FAS Unified Storage
FAS8080 EX
Hybrid and all-flash
Cloud integrated
FAS8060
SDS enabled
Massively scalable
FAS8040
– Scale up
FAS8020
– Scale out
FAS2554
FAS2552
FAS2520
2880TB
24TB Hybrid Flash
384TB All-Flash
8640TB
144TB Hybrid Flash
7200TB
384TB All-Flash
72TB Hybrid Flash
384TB All-Flash
4320TB
48TB Hybrid Flash
384TB All-Flash
864TB
16TB Hybrid Flash
154TB All-Flash
749TB
16TB Hybrid Flash
504TB
154TB All-Flash
16TB Hybrid Flash
96TB All-Flash
Unify and simplify storage administration under Data ONTAP®
FlexArray software lets FAS8000 manage EMC, HP, HDS, and E-Series
Clustered ONTAP: Data Agility & Always on
Virtualized Servers
Non-virtualized
Servers
Big Data
Content Servers
Workloads & Users
A
B
C
D
E
Policy-Based Data Management
Unexpected
Performance
Unexpected
Growth
Growth
Issue
Replication
All
Backup
Performance
Actions
Copy
Copy
Occur
Creation
Issue
Creation
Without
Capacity
System
Upgrade
Expansion
Needed
Needed
Growth
Issue
Resolved
in
Workload
D
E
B
Disruption
- -Workload
Workload
Resolved
to Users
A
C
Unified Data Network
FCP FCoE iSCSI
CIFS/SMB NFS/pNFS
Clustered ONTAP
A1
C1
53
Scalable Performance for SAN and NAS
Scale Out by Adding Controllers
(Maximum 24 Controllers)
Scale Up
Individual
Controllers
All-Flash FAS
All-Flash FAS
All-Flash FAS
All-Flash FAS
All-Flash FAS
All-Flash FAS
Hybrid FAS
Hybrid FAS
>4 million IOPS and nearly 5PB of all-flash capacity
NetApp® all-flash FAS: Scalable high performance for dedicated workloads
Mixed all-flash and hybrid FAS: One storage repository for all workloads
54
2014 NetApp,
All rights
reserved. .
© 2014© NetApp.
AlleInc.Rechte
vorbehalten
NetApp Flash & Storage Portfolio
Sweetspots for DP-targets
Sweetspots for ONTAP
snapshot-based DPR
SnapMirror/SnapVault targets
Sweetspots for Traditional DPR
sequential IO to DP targets
All Flash Storage
Hybrid Array
Sweetspots for VBR Backup
Deduplication Methods
Random-IO to DP targets
Flash-Accelerated Arrays
Traditional Storage
ONTAP
ONTAP
All Flash Aggregate
FlashRay
All Flash
E-Series
EF540
Flash Pool
Flash Cache
(in conjunction with Capacity disks)
SATA / SAS
Ennn
55
NetApp E-Series
An extreme fast & reliable Block
Storage (with less functionality
compared to Data ONTAP)
56
E-Series with Harddisks
Controllers + Shelf = Model Numbers
Controllers /
Disk Shelf
Enclosures
E2600
E2700
E5400
E5500
DE1600
(2U/12d)
E2612
E2712
E5412
E5512
E2624
E2724
E5424
E5524
E2660
E2760
E5460
E5560
3.5” only
DE5600
(2U/24d)
2.5” only
DE6600
(4U/60d)
2.5” or 3.5”
57
E-Series with Flash-only
EF550 Product Specifications
Base
models
12 drives (9.6TB)
24 drives (19.2TB)
Expandable
Up to 120 drives (96TB)
Drive
800GB, mixed-use, SSD
I/O
interface
options
Storage OS
Key
Features
(8) 16Gb FC
(8) 10Gb iSCSI
(8) 6Gb SAS
(4) 40Gb IB
SANtricity® 11.10
Dynamic disk pools
Thin provisioning
Snapshot™ copies
Volume copy
Remote mirroring
Performance:
• Burst: 900,000 IOPS
• Sustained: over 400,000 IOPS
• Sustained: up to 12GB/s
58
Kundenbeispiel DP Diskpool für NetBackup & TSM
Lokation 1
Lokation 2
5 Media Server
5 Media Server
NetApp E-Series
NetApp E-Series
NetApp E-Series
750 TB
750 TB
750 TB
750 TB
Kunden Infrastruktur
NetApp E-Series
Total: 3 PB
59
Storage Systems for Big Data
Physical Tape ?
60
Why is Disk a growing
& Tape a shrinking backup-media?
Unloaded physical Tape-Media have the lowest price per TB, but:
Data
Data
Disk drives are cheaper
than Tape Drives
Disk positions in
Milliseconds Tape in Minutes
Disk allows parallel Access for Backups & Restores
Disk-Surfaces operate dust-free - Tape-Media not
Disk can leverage RAID for Non-Disruptive-Operations – be aware that
magnetism-level shrinks over time & risk growths
Disk fits for Incremental Restores (direct access is mandatory for
Snapshots, DP-Deduplication & file-level Incremental Restores)
Disk fits for DR-Readyness (BLIF / dedupe allow Backups over small
bandwith – no wait for media to get full, Data may be direct accessible)
61
Where does Tape still fit?
practical
hints
for Full Tape sequential Writes - Read nearly Never
where Read Access-Times of many minutes are acceptable
where a tape gets mostly to streaming mode (~60 to ~380 MB/sec)
where partial Reads of Tape is very seldom (~ < 1 Read per 250 written
Media).
Typical remaining use cases for physical Tape:
Augmentation of disk-based 2nd-4th Gen. Data Protection (D2D2T):
Additional DR Copies of the disk stored backup / archive data
or for longtime retention (of monthly, quarterly or yearly backups)
(like NDMP Dump / SMTape with weeks of Snapshots)
Where Off-siding of DP Medias are a must – (for example: where even
Block-Level-Incremental-forever creates too much WAN-Traffic)
for fast large DB Backups (only as long RTO is acceptable).
62
Chapter „DP Methods / Trends
for Big Data”
General Overview
NDMP
DP Gen. 2: D2D(2T)
DP Gen. 3: D2D DP-Clients for blocklevel
Incremental Forever Backups
DP Gen. 4: DP Management-Software from IBM for
Array-snapshot-based DPR with ONTAP
63
How to Backup Big Data ?
DP Methods / Trends
64
Simplifying Data Protection
Fueling Convergence
Continuous
Availability
Disaster
Policy Based
Backup &
Recovery
Data Availability
Recovery
Archive &
Retrieval
Data Protection Market paradigm change
2000
2010
2020 DP world market revenues
DR
Prim
Prim.
DP
Prim
DP & DR
HA
HA & DR
DP
1st Generation DP: 2nd Generation DP:
Incremental / Full
Regular full
backups to Tape Backup2Disk/VTL
Separate techniques
for DP, DR and HA
(Dedupe- and NonDedupe targets offer only
limited improvements to
1st generation solutions)
3rd Generation DP:
Block-incrementalForever Backups
4th Generation DP:
Snapshot-based DP,
avoids data-movement
(often CBTF / Clientfor backups and restores
Dedupe; reduced
with Mirroring / Replication
backup xfer rates &
(DP, DR, HA and Test/Devslower Restores & Tape
needs can be combined)
copies)
Dedupe & compression benefits
Traditional DP result in very poor DP SLAs
(too much data & files have to be transferred)
start at Primary Data and
can be kept in backup copies
4th Generation Snapshot based Data Protection is the future for the DC,
3rd Gen is the future for client DPR & may be an alternate for backups “out of Array Snapshots”

66
Data to be transported with 1st Gen. DPR
Backup to & Restore from Tape
Application
/FileServers
with DPClient
DP MediaServer
Primary
Data
Storage
Physical Tape
(2 Copies usual)
67
Data to be transported with 1st Gen. DPR
LAN-free Backup to & Restore from Tape
DP-Server
Application
-Servers
with DPClient
Primary
Data
Storage
Physical Tape
(2 Copies usual)
68
Data to be transported with 2nd Gen. DPR
with traditional disktargets
DP MediaServer
Trad. DP-disktarget
Application
-Servers
with DPClient
Primary
Data
Storage
Physical Tape or
another DP Media
69
Data to be transported with 2nd Gen. DPR
with Target Deduplication
DP MediaServer
Application
-Servers
with DPClient
Deduped
DP-disktarget
Primary
Data
Storage
Physical Tape or
another DP Media
70
Data to be transported with 3rd Gen. DPR
by Client site Deduplication
Partial Dedupe
logic
DP DPServer
Application
-Servers
with DPClient
Deduped
DP-disktarget
Primary
Data
Storage
Physical Tape or
another DP Media
71
Data to be transported with 3rd Gen. DPR
by VMware CBTF?
DP MediaServer
Application
-Servers
with DPClient
Capacity reduced
DP-disktarget
Primary
Data
Storage
Physical Tape or
another DP Media
72
Data to be transported with 4th Gen. DPR
Array Snapshots with NDMP
DP MediaServer
Application
-Servers
with DPClient
Primary
Data
Storage
Nearly all Restores can be done
using the internal snapshots;
Deduplication/Compression may
Start on Primary
Physical Tape or
another DP Media
73
Array Snapshots with Replication &NDMP
DP MediaServer
Application
-Servers
with DPClient
Primary
Data
Storage
Start on Primary
Secondary
Storage in a
DR-usable
format
optional
Physical Tape or
another DP Media
74
Array Snapshots with Replication
DP MediaServer
Application
-Servers
with DPClient
Primary
Data
Storage
Secondary
Storage in a
DR-usable
format
Start on Primary
75
Which performance degrading can be
expected with Snapshots?
CofW at NetApp E-Series: Based on internal Tests of NetApp, we saw the
following with their CoFW-Snapshots (compared to normal mode
without Snapshot):
– With full Flash Arrays (SSD only): ~15% Throughput reduction
– With physical Disks (HDD only): ~40% Throughput reduction
Some other competitive Storage with CoFW: has been tested by 3rd
Party institutes with ~64% Throughput reduction with physical HDDs
but only ~3.2% Throughput reduction with ONTAP WA-based
Snapshots with physical disks and the same mix of IOs.
76
Snapshot Technique Use-Cases
Full Copy Snapshot arrays,
Typical RoW arrays & Software Snapshots (VMware, MS VSS etc)
basic Array-Snapshot-Support of some DP Software
every
Snapshot backup
1
4 to ~16
h to few days
Disk
stage
Tape
plus typically
Disk
stage
RoW Arrays with large Block,
CoFW Arrays
plus typically
Tape
(disk/cloud)
up to hundreds of Snapshots WA Appliances (like NetApp ONTAP)
weeks of Snapshots on Primary
(disk/cloud)
up to hundreds of Snapshots
months of Snapshots on Secondary
Tape
(disk/cloud)
77
4th Gen: Snapshot with Replication [+ NDMP]
without MetroCluster
practical
Block-Level-Incemental-Forever ~ 1,3% daily
hints
=für RoBo DP: bestens,
da lokale Restores extrem schnell;
Prim. Storage kleiner Niederlass.
SnapVault: Meist
werden hier
Snapshots
länger gehalten –
auf günstigen
SATAKapazitäts-Disks
Backup
DR falls Sec.-Storage noch verfügbar.: Server
D-RPO: ~1,3h bei 24 daily Backups
D-HWN: nur bis Prim.Storage / Server –
evtl. auch Aktivierung Sec.Storage möglich
D-RTO: Aktivierung oder alles zu übertragen,
aber schnelles 1:1 „Seeding“ möglich
Tape-Backups
werden optional &
sind rund um die Uhr
ohne Störung der
Produktion
möglich
SAN
Nur Differenz-blöcke
werden repliziert
Snapshot / SnapRestore
Server (evtl. mit
SnapDrive /
SnapManager)
FC-SAN /
iSCSI
Sehr geringe Netzwerklast
0%
NDMP Dump
or SMTape
Dedupe-/Compression-Vorteile
erhalten)
166 / 1,3%
69% / 0,8% densed
IP
WAN / LAN
NetApp Pri-Storage SnapVault oder SnapMirror
100 / 18%
1,3% or 0,8% densed
(SnapMirror/SnapVault kann
NetApp Sec-Storage
Physical Tape
Library
78
4th Gen: Snapshot with Replication [+ NDMP]
with MetroCluster
practical
Block-Level-Incemental-Forever ~ 1,3% daily
hints
iSCSI
DR falls Metro-Cluster-Hälfte verfügbar,
bleibt, da weit genug entfernt:
D-RPO: 0 Sekunden Datenverlust
D-HWN: Storage verfügbar
D-RTO
~1 Minute für Failover
FC-SAN /
SnapVault: Meist
werden hier
Snapshots
länger gehalten –
auf günstigen
SATAKapazitäts-Disks
Backup
Server
Tape-Backups
werden optional &
sind rund um die Uhr
ohne Störung der
Produktion
möglich
SAN
Sync
Mirror
Server (evtl. mit
SnapDrive /
SnapManager)
FC-SAN /
iSCSI
Nur Differenz-blöcke
werden repliziert
Sehr geringe Netzwerklast
NDMP Dump
or SMTape
Dedupe-/Compression-Vorteile
erhalten)
166 / 1,3%
69% / 0,8% densed
IP
0%
WAN / LAN
NetApp Pri-Storage SnapVault oder SnapMirror
100 / 18%
1,3% or 0,8% densed
(SnapMirror/SnapVault kann
NetApp Sec-Storage
Physical Tape
Library
79
NetApp Integrated Data Protection and
Business Continuity Solutions
Continuous
Operations
DR Readyness
Asynchronous
Replication
MetroCluster™
(full sync)
Fast Restores
SnapMirror®
Backup
out of
ONTAP
SnapVault®
SnapRestore®
Block-Level
Incremental
forever
Backups
Multiple
Recovery
Points per Day
NDMP backups
Cost
Availability
Snapshot™
Copies
Local Restore
Remote Restore
Remote Recovery – Low RTO
Capability
80
NDMP Dump und
SM2T / SMTape
(ONTAP Tapebackup)
As this chapter is a frequent subject of change, we offer our EMEA NetApp Partner community the access
to an ~3-monthly actualized version up to end of FY14 at:
https://fieldportal.netapp.com/search/?searchTerm=unterseher+NDMP# (after FY14 this will be accessible
for TechDemo Backup Deepdive attendees of Germany & Switzerland & Austria Sales-Partners only).
81
NDMP Modes
Local, 3-Way, Remote, filer2server
Local
NDMP
3-Way
NDMP
Remote
NDMP
(filer2filer
NDMP)
(filer2server
NDMP)
LAN
NDMP
Hosts
Data
+
Tape
Service
server2filer
NDMP
DMA +
Tape
Service
Data
Service
Tape
Service
DMA +
Data
Service
Data
Service
Tape
Service
DP
Clients
Automated
Tape Library
(PTL or VTL)
82
Tapebackup Technique Comparism
(current functionality with Data Ontap)
NDMP Engine and Ontap-Version
DUMP
SMTape (=“SM2T” with Ontap 7.x)
recommended backup use case
Tape DP Archival
Tape Disasterprotection
Objects backed up per volume?
Files & Directories
of one Snapshot
The full block construct of a volume
(including all snapshots & clones)
Full; filelevel Incr/Diff
Full; ONTAP 8.1.1: +blockl. Incr / Diff
single file / directory restore?
yes
no (only full volume restore)
Preservers dedupe/compression
No
Yes, benefits are kept (=faster)
Yes (no WAFL pointers)
no
limited (file-level)
high
Backupspeed Volumes with medium/large files
high
high
Is Clustered ONTAP already supported?
yes
no (expected in future)
Tape format backward support (Restorability)
Longtime (10 Years)
2 Ontap-Versions only (~3 Years)
format backward support (7- <--> C-Mode)
YES (not for LUNs)
no
no
no (no SnapMirror License)
Dump
SMTape (SM2T)
NDMP-Standard
NDMP-Extension type=snapm
separate CLI-Commands each
backup modes
Logical media break (=without WAFL-P.)
Backupspeed Volumes with millions of
small files
License costs NetApp?
Tape format
Protocol (NDMP DMA / CLI-Command)
83
supported NDMP Features
by Backup Application Vendors
a
Backup Application
D
DAR
ASG (Atempo) Time Navigator (>= 3.7)
Catalogic DPX (BEX) (>= 2.35)
Commvault Simpana (>=7.0)
CA BrightStor ARCserve (>= 16.0)
EMC / FTS NetWorker (>= 7.6.3)
HP Storage Data Protector (>=6.2)
c
d
f
g
s
t
u
3W
DDS
remote
f2s
s2f
SMTape
full
SMTape
Incr/Diff
CAB
4.3.
Soon exp.
8.0
b restartable
Backup
Oracle Secure Backup (>= 10.3.0.2)
?
?
?
?
NetApp SnapProtect (>=9.0SP3b)
expected
?
?
?
?
IBM Tivoli Storage Manager (>= 5.5.2)
Quest (BakBone) NetVault (>= 8.0)
Symantec NetBackup (>=5.1)
Symantec BackupEXEC (>= 12.5)
Zmanda Amanda Enterprise (>= 7.3.2)
Direct Access Recovery (DAR), and Local Backup is supported by all of them
a = Directory DAR
*2? *2? *1 *2 *2
*1 *2
?
?
*1*2 *2
*2? *1 *2
*2?
?
?
V10SP2b V10SP2b *3
?
V10SP2b V10SP2b *3
7.6.7+patch
?
c – 3-way NDMP (filer filer)
d = Dynamic Drive Sharing (DDS)
f – Remote NDMP (filer media server), with *1 = Tape-Multiplexing optional, with *2 = disk-Caching optional
g – Server-to-Filer Backup
s – SM2T / SMTape full backup/restore
t – SMTape Incremental or Differential backup
u=cluster aware backup *3= currently without local NDMP (soon expected)
84
ONTAP Releases supported for NDMP
by Backup Application Vendors
ONTAP 8.0
ONTAP 8.1
ONTAP 8.2
(7-mode & Clustered
ONTAP)
(7-mode &
Clustered ONTAP)
(7-mode &
Clustered ONTAP)
Backup Application
ONTAP
7.3
ASG (Atempo) Time Navigator
4.2
4.2
Catalogic DPX (BEX)
3.4.1, 4.1
3.4.1, 4.1
3.4.1, 4.1
4.2
Commvault Simpana
8.0
8.0
9.0SP5
9.0, 10.0
CA BrightStor ARCServe
11.5, R15
R15, R16 (7-Mode only)
R16 (7-Mode only)
R16.5
EMC NetWorker
7.4
7.5 7.6 (7-mode only)
7.6.3, 8.0
8.0
HP Storage Data Protector
6.0
6.11, 6.2 (7-mode only
6.21, 7.01 (7-mode only
without DAR-Feature)
without DAR-Feature)
9.0.1 (both modes)
NetApp SnapProtect
9.0SP3b
9.0SP3b
9.0SP5
Oracle Secure Backup
10.3.0.2
IBM Tivoli Storage Manager
5.51, 6.1
6.1
Quest(BakBone) NetVault
8.2
NVBU 8.5.3
9.0
9.x (cDOT only)
Symantec NetBackup
6.0 & 7.0
6.5 & 7.0
7.1 & 7.5
7.5
Symantec BackupEXEC
12.5
2010R2(+ Hotfix)
2012
2012 SP4
Zmanda Amanda Enterprise
3.1
4.3.3
5.5.x,
6.1.x
V10 SP2b
6.3.3
NetApp IMT http://support.netapp.com/matrix is most actual
or use official certification side: http://www.netapp.com/us/solutions/a-z/backup-to-tape/backup-to-tape-ndmp.html
85
DP Generation 2:
D2D(2T)
(mit gelegentlichen Fullbackups)
86
Allgemeines zu DP Gen 2: D2D(2T)
Future...
Gen 2 DP: Sie haben die Wahl zwischen folgenden DP-Disktargets:
2a) DP-disktargets ohne Verdichtung (z.B. mit der NetApp E-Series):
Ist aufgrund des hohen Diskbedarfs dann sinnvoll, wenn große Fullbackups nur wenige
Tage oder nur auf Tape gehalten werden.
NetApp‘s E-Series ist dafür ab ~130 TB Kapazität performant & wirtschaflich (~3
GB/sec sequ. Write / ~5 GB/sec sequ. Read, wenig Stromverbrauch).
2b) DP-disktargets mit Compression only (z.B. ONTAP ab 8.1):
Hier kann man (bei Reduzierung der Fulls) oft bis zu ~ 4 Wochen Backups auf Disk
vorhalten.
ONTAP sollte typischerweise bis max ~50 TB Kapazität auch für Tape-Prozesse
performant genug sein.
2c) DP-disktargets mit Deduplication (z.B. SteelStore, impliziert Compression):
Erlaubt oft bis max ~3 Monate Backups auf Disk zu halten.
Kann bei höheren Dedupe-Ratios sinnvoll sein, falls die Dedupe-Nachteile akzeptabel
bleiben und (gewünschte) Tape-Kopien noch genügend skalieren.
87
Allgemeines zu DP Gen 2: D2D(2T)
practical
hints
Ihre Backups sollten virensicher sein!
Wenn Backups nur auf Disk-/NAS-Targets gespeichert
werden (ohne Kopie auf PTL/VTL), kann über Viren/Hacker ein Totalverlust der Backups eintreten denn ein Virus der in den DP-Server eindringt kann sehr
einfach alle im Schreibzugriff befindlichen Files
zerstören.
Schnittstellen wie VTL bieten hier einen besseren
Schutz.
Falls Sie ONTAP als Disktarget verwenden,
könnten zusätzlich aktivierte ONTAP Snapshots
helfen, die Backup-Files für ~1-3 Tage vor Viren
etc zu schützen (=höherer Platzbedarf).
DP-target Appliances, welche auf Standard-OS
(wie RedHat Linux) basieren, können zusätzlich
ein Ziel von Hackern / Viren werden, welche
Lücken im OS nutzen.
88
DP Generation 3
D2D DP-Clients für Incremental
Forever Backups
89
DP Gen 3: Incremental Forever DP Clients,
Überblick
Gen 3 DP: Incremental Forever (Full nur zu Beginn) gibt es in 3
Ausprägungen von DP-Clients:
3a) Filelevel IF (TSM-like oder mit Synthetic Fulls des DPServers)
Nur begrenzt effizient, vor allem ungünstig bei kleinen
Änderungen großer Applikations-Files (wie DB-Files).
Also ist diese Technik eigentlich nur „DP Generation 2.5“.
3b) Client side Deduplication (ergibt auch einen BLIF Effekt)
3c) „echtes“ Blocklevel IF (z.B. Catalogic DPX / SPOS / OSSV
/ VMware with changed Block Tracking forever)
More hidden
90
DP Gen 3b: client side Deduplication
Funktionsweise / Stärken / Schwächen
Ein Teil der Dedupe-Logik ist auf den DPClient verlagert
arbeitet immer mit dem kleinen
HashKeyCollission Risiko
Der Backup-Datenstrom ist relativ gering
…
91
3c: VMware vStorage API-basierte Backups
mit Changed Block Tracking Forever
Ist nur für VMware Datastores (VMDKs) anwendbar und nur für die neuen
VMware Versionen ab ESX V5.
Merkt sich Blockänderungen (z.B. auf 16K Ebene) für den Folge-Backup.
Gibt diese Änderungen per API an eine Backup-Software
CBTF unterstützen typischerweise erst die neuesten DP-Software
Versionen seit ~Ende 2013 (z.B. TSM).
Die Backup-Software muss hier teilweise komplexe Dinge ergänzen
–
–
–
–
–
Z.B.: für logical Object Restores das komplette VMDK bereit stellen
Expiration (Löschen) ausgealterter Backups
Fragmentierung für Full VMDK-Restores sollte gering gehalten werden Speed
(oft wird dann doch wieder mal ein echter Fullbackup ausgeführt)
Störungen müssen erkannt werden (meist wird dann ein erneuter Fullbackup
ausgelöst)
etc
92
DP Management-Software
from IBM
for Array-snapshot-based
DPR with ONTAP
93
IBM TSM NAS Data Protection mit ONTAP (1)
Folgende Kombination ist für „Millions-of-Files“ NAS Volumes (im Vergleich zum
normalen TSM-Workflow) vorteilhaft:
SM2T / SMTape seit TSM Server 5.5.2: für schnelles DR Backup/Restore
(Differential seit ONTAP 8.1.1: TSM Support dafür gefordert / noch nicht absehbar).
Plus SnapDiff (7-Mode ony!): für beschleunigten TSM Scan-Prozess beim Backup
(mehr Details dazu auf Folgeseite)
Noch effizienter für „Millions-of-Files“ NAS Daten ist aber meist:
mehrwöchig gehaltene NetApp-erzeugte Snapshots,
ergänzt um NetApp-gesteuerte SnapMirror/SnapVault-Replikation
und / oder TSM-gesteuertem NDMP SMTape / Dump.
practical
hints
94
IBM TSM NAS Data Protection mit ONTAP (2)
SnapDiff (7-Mode): für beschleunigten TSM Scan-Prozess beim Backup:
ONTAP ab 7.3.3 / 8.0.1 & TSM Client ab 6.2.2.2 auch für deutsche Umlaute:
CIFS über Windows-Client, NFS über AIX- oder Linux-Client
seit TSM-Client 6.40 & ONTAP 8.1.1 vfiler-Multistore Support
Seit TSM-Client 6.40 auch für bestehende Snapshots / auch von
SnapVault/SnapMirror targets
noch kein Clustered ONTAP Support von Seiten IBM
noch kein V-Series Support (obwohl es funktionieren sollte)
IBM empfiehlt bei SnapDiff-Verwendung trotzdem gelegentlich (z.B: einmal im
Monat / Quartal) einen traditionellen TSM Full Incremental Backup (also ohne
SnapDiff) durchzuführen.
practical
hints
In dem Fall würden nur evtl. entstehende Lücken des SnapDiff-beschleunigten Backup
gefüllt. IBM hat bisher nicht beschrieben wieso – Dieter Unterseher hat folgende
Vermutungen:
Vielleicht fehlen Incremental Backups nach ACL Rechteänderungen auf Directory
Ebene, wenn SnapDiff angewendet wird?
Vielleicht kann es bei der TSM-Logik SnapDiff vorkommen, dass ein nicht zu Ende
gebrachter SnapDiff-Backup zu Backup-Lücken führen kann.
95
IBM TSM / FCM snapshot-based DP mit ONTAP
ONTAP Support für Snapshot/SnapRestore (ohne ReplikationsKontrolle) verfügbar seit 16.11.2012 mit FCM 3.2 (mit TSM 6.40; ONTAP
7-Mode only):
Oracle / DB2 / SAP(auf Oracle/DB2) unter AIX & Linux (native):
– blockdevice support (mit schnellem Full Volume SnapRestore)
Microsoft SQL & Exchange (native)
(seit 3.2.1 auch Exchange2013 supported):
– sehr schnelle Backups;
– logical Object Restores aus gemounteten Snapshot-Kopien
VMware: VMDK-Level Backups per Snapshot für block- & NFS Data Stores:
– Und Application Consistent Backups für MS SQL / Exchange etc (Object
Restores mit VMware & MS VSS Tools)
– Bei NFS Datastores sind auch sehr schnelle VMDK-Level Restores (über
Single File SnapRestore) möglich
96
IBM TSM / FCM snapshot-based DP mit ONTAP
IBM-Infos dazu:
http://www-01.ibm.com/software/tivoli/products/storage-flashcopy-mgr/
NetApp-Infos dazu:
FAQ „Tivoli Storage FlashCopy Manager Version 3.2”
vom NetApp Field Portal
Fundierte Umsetzungs-Erfahrung liegt z.B. vor:
EMPALIS (da kann man das auch live mit ONTAP vorführen
lassen); siehe http://www.empalis.de/index.php?id=2308&L=)
97
TechDemo Serien
practical
hints
www.techdemo.de
zum Thema „Backup & Recovery“ : siehe
http://techdemo.de/events/kat/backup.html
Neben der Ganztagesveranstaltung
„TechDemo Backup Deepdive“ bietet NetApp seit
Sommer 2013 für IT-Manager & Sales-Beauftragte
von Partnern die Halbtagesveranstaltung
„TechDemo Backup – Überblick“ und
gelegentlich Partner-Veranstaltungen;
Ergänzend können Sie immer mehr Einzelthemen
per TechDemo Online bei Bedarf abrufen:
http://techdemo.de/online.html
98
Ende
Offene Fragen?
© 2013 NetApp, Inc. All rights reserved. No portions of this document may be reproduced without prior written consent of NetApp, Inc. Specifications are
subject to change without notice. NetApp, the NetApp logo, Go further, faster, AutoSupport, Data ONTAP, OnCommand, SnapDrive, SnapManager,
SnapMirror, Snapshot, and SnapVault are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. Microsoft,
SharePoint, SQL Server, Windows, and Windows Server are registered trademarks and Hyper-V is a trademark of Microsoft Corporation. Linux is a
registered trademark of Linus Torvalds. Oracle is a registered trademark of Oracle Corporation. ESX and VMware are registered trademarks and ESXi is a
trademark of VMware, Inc. UNIX is a registered trademark of The Open Group. All other brands or products are trademarks or registered trademarks of their
respective holders and should be treated as such.
99

Vortrag

Transcrição

Documentos relacionados

Schmitz-Werke erweitern NetApp

USV-Anlagen bei Kess Power Solutions

Alle Daten griffbereit Ziemann Group sorgt mit NetApp für

C. Rob Hammerstein stützt virtualisierte IT-Infrastruktur von

Self-Sealing Tape

Techno AG - Technisches Datenblatt Innotec Self

highlights - Sky Media

Backup - ulibauer.de

highlights - Sky Media

NetApp™ SupportEdge Standard Leistungsbeschreibung