Immersive Medien und 3D-Video

Transcrição

Fraunhofer-Institut für
Nachrichtentechnik
Heinrich-Hertz-Institut
Ralf Schäfer
[email protected]
http://ip.hhi.de
page 1
Immersive Medien
und 3D-Video
R. Schäfer - Immersive Medien und 3D-Video
Outline
page 2
 Immersive Media
 Examples
 Interactive Media and Free Navigation
 Free Navigation in Video Scenes
 Examples
 Free Viewpoint Video
 Summary
Immersive Media – Costs vs. Sensation
Costs
Real Live Sensation
American
Football
IMAX
Rock Concert
Theatre
Cinema
VoD
VCR
CD
TV
Interactive
TV
Sensation
page 3
Immersive
Media
page 4
Immersive Display (UHDTV, D-Cinema)
Example: Immersive Video Conferencing – im.point
Cameras
61” Plasma
display
Audio Speakers
• Seamless transition between the real and virtual world
• Head motion parallax
• Life-sized upper body images representing gestures and body language
• 3D representation of the remote participants including provision of eye contact
page 5
Semi-circular
table
Example: Largest Panorama Worldwide
Gasometer
in Leipzig
and 36 m hight (weight > 1 ton)
page 6
Printed panorama with 60 m diameter
page 7
Cylindrical Projection with HHI‘s CineCardultra
Single l e v e l
Multiple levels
 Horizontal & vertical soft edge blending
υ
page 8
CineCardultra:
 Warping for cylindrical and parabolic projection
υ
Interactive Media

Allow to be active in some way instead of just
being a passive consumer

Enabled through convergence on
telecommunications, consumer electronics and
computer science

Well-known: linking of multimedia data for
interactive access
Multimedia DVD, Internet presentations,
Multimedia Home Platform (MHP)
page 9

Free Navigation

Arbitrary selection of viewpoint and direction within real
world audio-visual scenes

Well-known from

computer graphics, virtual reality, computer games
synthetic scenes, objects or static views of real
objects

or restricted to predefined viewpoints (e.g. Formula 1
on Premiere)
page 10

Free Navigation in Real Scenes

New technology allows generation of intermediate
views in between real existing camera view

Image-Based rendering, 3D reconstruction

Allows e.g. viewing sports or theatre from arbitrary
viewpoints

New technologies for the whole chain:
Acquisition, signal processing, modelling,
representation, compression, transmission,
rendering, display
page 11

Scene Representations
More geometry -based
Static
Texture,
Single
Geometry
View-dep.
Texture,
Single
Geometry
More image-based
Static
Texture,
View-dep.
Geometry
View-dep.
Texture,
View-dep.
Geometry
Incarnations of 3D model
Sprites
Warping
Light-field
rendering
Lumigraph
Interpolation
page 12
Conventional graphics pipeline
Layered Depth
images
Omni-directional Video

Systems with curved mirrors
www.remotereality.com
page 13

Omni-directional Video – Example 1
Courtesy of Prof. Yokoya, Nara Institute of Science
and Technology
page 14

page 15
Omni-directional Video – Example 2
Interactive Navigation in 3D Panoramas
page 16
Patches
Creation of 3D Panoramas
 By using several shots of a still camera and stitching techniques
page 17
 By using video and mosaiquing techniques
Interactive Decoding and Rendering
Tiles can be streamed as
JPEG or JPEG2000 files
Visibility sensor
page 18
Current view on screen
Free Viewpoint Video (FVV)



Capturing a real dynamic scene with N cameras
Transformation into a special data representation
Interpolation of arbitrary intermediate views
Allows free navigation within the scene (within
practical limits)
page 19

Example: Eye Vision using tracking cameras
Virtual
Camera
Path
page 20
X
Eye Vision



No reconstruction of 3D geometrie => many cameras
No individual interaction possible
Live broadcast event: Super Bowl 2001, 30 cameras
Courtesy of Prof. Takeyo Kanade, Carnegie Mellon Univ.
page 21

Visual Hull Reconstruction
Background subtraction
Silhouette extraction
Visual Hull
reconstruction
page 22
Synchronized
multi-video recording
page 23
Example: Temple Acquisition by Photographs
page 24
Processing Step: 3D Mesh Extraction
page 25
Processing Step: Texturing
Multi camera video acquisition
page 26
Courtesy of
Stephan Wuermlin
et al.,
ETH Zuerich,
Switzerland
Segmentation
page 27
Courtesy of Stephan
Wuermlin et al.,
ETH Zuerich,
Switzerland
Source: ETH Zürich
page 28
Original sequence (1 camera)
Free Viewpoint Video from 16 Cameras
Results produced by HHI

Same functionality as CG
objects
 free navigation, can be
viewed from any
viewpoint/direction
 Integration into complete
scenes
(virtual/augmented/real)

But: depict appearance,
motion, deformation of real
world objects
page 29

Courtesy of Microsoft Research, Redmont, WA, USA
page 30
Example: Depth-based Rendering
Conclusion
page 31
 Immersive Media is a challengig and exciting
field of research & development
 There are many approaches to allow
interactivity and free navigation in video scenes
 Free Viewpoint Video (FVV) is a new concept
for video entertainment which enables the user
to navigate in real scenes in the same way as
in virtual worlds
Thank you
Fraunhofer Heinrich-Hertz-Institut
page 32
We put science into action

Immersive Medien und 3D-Video

Transcrição

Documentos relacionados

CineGrid International Workshop 2014 Program