From Delphi… t Phi T …to Phi-T - Indico

From Delphi…
…to
t Phi
Phi-TT
Spin-Off from Physics Research to Business
Prof. Dr. Michael Feindt
KCETA - Centrum für Elementarteilchen- und Astroteilchenphysik
IEKP, Universität Karlsruhe, Karlsruhe Institute of Technology
gy KIT
& Phi-T GmbH, Karlsruhe
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting
DELPHI Closing Meeting, CERN
CERN, May 29, 2008
May 2008
HPC
C ((1991-92))
Initially energy and spatial
resolutions catastrophic
SSolve
l problems
bl
bby llooking
ki
IN DETAIL (144 modules )
into the real DATA.
Correct by software
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
HPC
C
Æ design
g pperformance almost reached,, pphysics
y
analysis
y possible
p
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Introduction off S
Software
f
Tasks
e.g.
g
ELEPHANT,
MAMMOTH
packages
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
(
(Inclusively
reconstr.)) B physics
B*
From DELPHI to Phi-T
Michael Feindt
B**
DELPHI Closing Meeting, CERN
May 2008
Start to work with neural networks (in1993)
BSAURUS: b flavour tagging
gg g
From DELPHI to Phi-T
Michael Feindt
MACRIB: Combined Particle ID
DELPHI Closing Meeting, CERN
May 2008
Gained
G
i d lots
l t off experience
i
with
ith neurall network
t
k
applications in DELPHI (1993-2003)
Electron id (ELEPHANT)
Kaon, proton id (MACRIB)
(
(BSAURUS::
Optimisation of resolutions inclusive B- E, , , Q-value
Inclusive B production flavour tagging
Inclusive B decay flavour tagging
Inclusive B+/B0/Bs/Lambda_b separation
B**, Bs** enrichment
B fragmentation function
Li it on Bs-mixing
Limit
i i
B0-mixing
B- F/B-asymmetry
B > wrong sign charm)
B->
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Ne ral reconstruction
Neural
reconstr ction of a direction
Resolution of azimuthal angle of inclusively
reconstructed B-hadrons in the DELPHIdetector
first neural reconstruction of a direction
N
NeuroBayes
B
phi-direction
Best
”classical"
chi**2- fit
(BSAURUS)
No selection:
Improved resolution
From DELPHI to Phi-T
Michael Feindt
After selection cut on estimated error:
R l i massively
Resolution
i l improved,
i
d no tails
il
==> allows reliable selection of good events
DELPHI Closing Meeting, CERN
May 2008
N ï neurall networks
Naïve
t
k and
d criticizm
iti i
We‘ve tried that but it didn‘t give good results
- Stuck in local minimum
- Learning not robust
We‘ve
We
ve tried that but it was worse than our 100 person
person-years
years
analytical high tech algorithm
- Selected too naive input variables
- Use
U your fancy
f
algorithm
l ith as INPUT !
We‘ve tried that but the predictions were wrong
- Overtraining: the net learned statistical fluctuations
Yeah but how can you estimate systematic errors?
- How can you with cuts when variables are correlated?
- Tests on data, data/MC agreement possible and done.
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Address all these topics and build a
professional robust and flexible neural
network package for physics and general
prediction
di ti tasks:
t k
NeuroBayes®
(2000-2002)
(2000
2002)
turned out to be a very strong tool
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
After having
ha ing looked outside
o tside the
ivory tower, realized:
These methods are
not only applicable
in physics
<phi-t>:
From DELPHI to Phi-T
Foundation of private company out of University
of Karlsruhe, initially sponsored by exist-seedprogramme of the federal ministery for Education
and Research BMBF
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
History
2000-2002 NeuroBayes®-specialisation
for economy at the University
of Karlsruhe
Oct. 2002: GmbH founded,
fi
d t i l projects
j t
firstt iindustrial
June 2003: Removal into new office
199 sqm
q IT-Portal Karlsruhe
May 2008: Expansion to 700 sqm office
Foundation of sub-company
Phi T products&services
Phi-T
prod cts&ser ices
Exclusive rights for NeuroBayes®
Staff all pphysicists (almost all from HEP)
Continuous further development of NeuroBayes®
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Customers:
(among others):
BGV and VKB car insurances
Lupus Alpha Asset Management
Otto Versand (mail order business)
Thyssen Krupp (steel industry)
From DELPHI to Phi-T
Michael Feindt
AXA and Central health insurances
dm drogerie markt (drugstore chain)
Libri (book wholesale)
DELPHI Closing Meeting, CERN
May 2008
Input
Ne roBa es® principle
NeuroBayes®
NeuroBayes® Teacher:
Learning of complex
relationships from existing
data bases (e.g.
(
Monte Carlo))
NeuroBayes® Expert:
Prognosis for unknown data
Significcance co
ontrol
Preprocessing
Postprocessing
Output
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Ho it works:
How
orks training and application
Historic or simulated
data
Data set
a = ...
b = ...
c = ...
....
t = …!!
NeuroBayes®
N
B
®
Teacher
Expert system
Probability that hypothesis
is correct ((classification))
or probability density
for variable t
Expertise
Actual (new real) data
Data set
a = ...
b = ...
c = ...
....
t=?
From DELPHI to Phi-T
NeuroBayes®
Expert
Michael Feindt
f t
DELPHI Closing Meeting, CERN
t
May 2008
NeuroBayes®
Ne
roBa es® task 1
1:
Classifications
Classification:
Binary targets: Each single outcome will be “yes“ or “no“
NeuroBayes output is the Bayesian posterior probability that
answer is
i “yes“
“ “ ((given
i
that
h iinclusive
l i rates are the
h same in
i training
i i
and test sample, otherwise simple transformation necessary).
Examples:
> This elementary particle is a K meson.
> This event is a Higgs candidate.
> Germany will become soccer world champion in 2010.
2010
> Customer Meier will have liquidity problems next year.
> This equity price will rise.
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
NeuroBayes®
Ne
roBa es® task 2
2:
Conditional probability densities
Probability density for real valued targets:
For each possible (real) value a probability (density) is given.
From that all statistical quantities like mean value, median, mode,
standard deviation, percentiles etc can be deduced.
Examples:
p
> Energy of an elementary particle
(e.g a semileptonically decaying B meson with missing neutrino)
> Q value of a decay
> Lifetime
Lif i
off a decay
d
> Price change of an equity or option
> Company turnaround or earnings
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Prediction of the complete probability
probabilit
distribution – event by event unfolding -
r
f (t | x )
E
Expectation
t ti value
l
Mode
Standard deviation
volatility
Deviations from
normal distribution,
e.g. crash probability
t
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Conditional probability
probabilit densities
in particle physics
What is the probability density
of the true B momentum
in this semileptonic B candidate event
taken with the CDF II detector
t
with these n tracks with those momenta and
rapidities in the hemisphere,
hemisphere
which are forming this secondary vertex with
this decay length and probability, this invariant
mass andd transverse
t
momentum,
t
thi
this llepton
t
information, this missing transverse momentum,
this difference in Phi and Theta between
momentum sum andd vertex topology,
l
etc pp
r
x
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
r
f (t | x )
May 2008
<phi t> NeuroBayes®
<phi-t>
Ne roBa es®
> is based on neural 2nd generation algorithms,
Bayesian regularisation,
regularisation
optimised preprocessing with transformations
and decorrelation of input variables and
linear correlation to output.
output
> learns extremely fast due to 2nd order methods and
0-iteration mode
> can train
t i with
ith weights
i ht andd background
b k
d subtraction
bt ti
> is extremly robust against outliers
> is immune against learning by heart statistical noise
> tells
t ll you if there
th
is
i nothing
thi relevant
l
t to
t be
b learned
l
d
> delivers sensible prognoses already with small statistics
> has advanced boost and cross validation features
> is
i steadily
dil further
f h developed
d l d
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Ramler-plot (extended correlation matrix)
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Ramler-II-plot (visualize correlation to target)
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Visualisation of single
g input-variables
p
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Visualisation of correlation matrix
Variable 1: Training target
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Visualisation of network performance
p
Purity vs. efficiency
Signal-effiziency vs.
total efficiency
(Lift chart)
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Visualisation of NeuroBayes
y network topology
p gy
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
S
Successful
f l applications
li ti
in
i High
Hi h Energy
E
Physics:
Ph i
att CDF II:
II
Electron ID, muon ID, kaon/proton ID
Optimisation of resonance reconstruction in many channels
(X Y
(X,
Y, D
D, Ds , Ds**,
** B,
B Bs, B**,B
B** Bs**)
Spin parity analysis of X(3182)
Inclusion of NB output in likelihood fits
B-tagging for high pt physics (top,
(top Higgs,
Higgs etc
etc.))
Single top quark production search
Higgs search
B-Flavour
B
Flavour Tagging for mixing analyses (new combined tagging)
B0, Bs-lifetime, DG, mixing, CP violation
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
F th N
Further
NeuroBayes®
B
® applications
li ti
in
i HEP:
HEP
AMS:
Particle ID design studies
CMS (ongoing) :
B-tagging
Belle (ongoing) :
Continuum suppression
H1 (ongoing) :
Hadron calorimeter energy resolution optimisation
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
M
More
th
than 35 di
diploma
l
and
d Ph
Ph.D.
D theses…
th
from experiments DELPHI, CDF II, AMS and CMS
used NeuroBayes® or predecessors very successfully.
Many of these can be found at
www.phi-t.de
p
Æ Wissenschaft Æ NeuroBayes
Talks about NeuroBayes® and applications:
www-ekp.physik.uni-karlsruhe.de/~feindt
k h ik i k l h d / f i dt Æ Forschung
F
h
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
J t a ffew examples…
Just
l
NeuroBayes soft electron identification for CDF II
(Ul i h Kerzel,
(Ulrich
K
l Michael
Mi h l Milnik,
Mil ik M.F.)
MF)
Thesis U.
U Kerzel:
on basis of Soft Electron Collection
(much more efficient than
cut selection
or JetNet with same inputs)
- after clever ppreprocessing
p
g byy hand
and careful learning parameter
choice this could also be as good
as NeuroBayes®
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
J t a ffew examples…
Just
l
NeuroBayes® selection
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
J t a ffew examples…
Just
l
First observation of B_s1
and most precise of B_s2*
Selection using NeuroBayes®
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Presse ((Die Welt vom 21. April
p 2006))
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Successful
in competition
with other
data-miningmethods
World‘s largest students
competion:
D t Mi i C
Data-Mining-Cup
2005: Fraud detection
in internet trading
2006: price prediction
in ebay auctions
2007: coupon redemption
prediction
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Start
Idee
NeuroBayes
Idee
Hintergrund
Ziele
NeuroBayes
f(t|x)
Beispiele
Historie
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Forschung
Projekt ll
Ablauf
Spiel
Summary
A
B
A
Phi-T: Applications of NeuroBayes® in Economy
> Medicine and Pharma research
e g effects and undesirable effects of drugs
e.g.
early tumor recognition
> Banks
e g credit
e.g.
credit-scoring
scoring (Basel II)
II), finance time series
prediction, valuation of derivates, risk minimised
trading strategies, client valuation
> Insurances
e.g. risk and cost prediction for individual clients,
probability of contract cancellation, fraud recognition,
justice in tariffs
> Trading chain stores:
turnover prognosis for individual articles/stores
Necessary p
prerequisite:
q
Historic or simulated data must be available!
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 200837
Start
Idee
NeuroBayes
Idee
Hintergrund
Ziele
NeuroBayes
f(t|x)
Beispiele
Historie
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Forschung
Projekt ll
Ablauf
Spiel
Summary
A
Individual risk prognoses
for car insurances:
Accident probability
C t probability
Cost
b bilit di
distribution
t ib ti
Large damage prognosis
Contract cancellation prob.
prob
Î very successful at
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 200838
B
A
Start
Idee
NeuroBayes
Idee
Hintergrund
Ziele
NeuroBayes
f(t|x)
Beispiele
Historie
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Forschung
Projekt ll
Ablauf
Spiel
Summary
A
B
A
The ‘‘‘‘unjustice‘‘
nj stice‘‘ of ins
insurance
rance premiums
premi ms
An
nzahl Kun
nden
Ratio of the accident risk calculated using NeuroBayes®
to premium paid (normalised to same total premium sum):
The majority
j y of customers (with
(
low
risk) are paying too much.
Less than half of the customers (with
larger risk) do not pay enough, some
by far not enough.
These are currently subsidised by the
more careful
f l customers.
t
Risiko/Prämie
/
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 200839
Start
Idee
NeuroBayes
Idee
Hintergrund
Ziele
From DELPHI to Phi-T
NeuroBayes
f(t|x)
Beispiele
Historie
Michael Feindt
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Forschung
Projekt ll
Ablauf
Spiel
DELPHI Closing Meeting, CERN
Summary
A
May 200840
B
A
Start
Idee
Turnover prognosis for mail order business
NeuroBayes
Idee
Hintergrund
Ziele
From DELPHI to Phi-T
NeuroBayes
f(t|x)
Beispiele
Historie
Michael Feindt
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Forschung
Projekt ll
Ablauf
Spiel
DELPHI Closing Meeting, CERN
Summary
A
May 200841
B
A
Start
Idee
NeuroBayes
Idee
Hintergrund
Ziele
NeuroBayes
f(t|x)
Beispiele
Historie
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Forschung
Projekt ll
Ablauf
Spiel
Summary
A
Presse
Physicists modernise sales management …
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 200842
B
A
Start
Idee
NeuroBayes
Idee
Hintergrund
Ziele
NeuroBayes
f(t|x)
Beispiele
Historie
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Forschung
Projekt ll
Ablauf
Spiel
Summary
A
Test
Test
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 200843
B
A
Start
Idee
NeuroBayes
Idee
Hintergrund
Ziele
NeuroBayes
f(t|x)
Beispiele
Historie
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Forschung
Projekt ll
Ablauf
Spiel
Summary
A
B
A
Prognosis
g
of individual health costs
Pilot project for a large private
health insurance
Prognosis of costs in following
year for each person insured
with confidence intervals
4 years of training, test on
following year
Kunde N. 00000
Results:
Mann, 44
Probability density for each
customer/tarif
/ f combination
Tarif XYZ123 seit
ca. 17 Jahre
Very good test results!
Has potential for a real and objective cost reduction in health management
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 200844
Start
Idee
NeuroBayes
Idee
Hintergrund
Ziele
NeuroBayes
f(t|x)
Beispiele
Historie
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Prognosis
g
of financial markets
Forschung
Projekt ll
Ablauf
Spiel
Summary
A
VDI-Nachrichten, 9.3.2007
NeuroBayes® based risk averse market
neutral fonds for institutional investors
Lupus Alpha NeuroBayes® Short Term
Trading Fonds (up to now 20 Mio €,
aim: 250 Mio
Test € end of 2008)
Test
Börsenzeitung, 6.2.2008
From DELPHI to Phi-T
Michael Feindt
B
A
DELPHI Closing Meeting, CERN
May 200845
Start
Idee
NeuroBayes
Idee
Hintergrund
Ziele
NeuroBayes
f(t|x)
Beispiele
Historie
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Forschung
Projekt ll
Ablauf
Spiel
Summary
A
B
A
The <phi-t> mouse game:
or:
even your ``free
``f
will´´
´´ is predictable
//www.phi-t.de/mousegame
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 200846
Bindings and Licenses
NeuroBayes® is commercial software.
Special rates for public research.
research
Essentially free for high energy physics research.
License files needed. Separately for expert and teacher.
Please contact Phi-T.
y core code written in Fortran.
NeuroBayes
Libraries for many platforms (Linux, Windows, …) available.
Bindings exist for C++, Fortran, java, lisp, etc.
Code generator for easy usage exists for Fortran and C++.
New: Interface to root-TMVA available (classification only)
Integration into root planned
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Doc mentation
Documentation
Basics:
M Feindt,
M.
Feindt A Ne
Neural
ral Bayesian
Ba esian Estimator for Conditional
Probability Densities, E-preprint-archive physics 0402093
M. Feindt, U. Kerzel, The NeuroBayes Neural Network Package,
NIM A 559(2006) 190
Web Sites:
www.phi-t.de (Company web site, German & English)
www neurobayes de (English site on physics results with
www.neurobayes.de
NeuroBayes & all diploma and PhD theses using NeuroBayes)
www-ekp.physik.uni-karlsruhe.de/~feindt (some NeuroBayes
talks can be found here under -> Forschung)
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 2008
Start
Idee
NeuroBayes
Idee
Hintergrund
Ziele
NeuroBayes
f(t|x)
Beispiele
Historie
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Forschung
Projekt ll
Ablauf
Spiel
Summary
A
B
A
At DELPHI I have learned how to extract
information and learn from data.
With a physicist’s pragmatic analytical and
quantitative thinking, data-analysis and
programming expertise and NeuroBayes®, we
h
have
achieved
hi d a lot
l t in
i physics
h i andd
with Phi-T in very different areas of industry
and economics.
economics
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 200849
Start
Idee
NeuroBayes
Idee
Hintergrund
Ziele
NeuroBayes
f(t|x)
Beispiele
Historie
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Forschung
Projekt ll
Ablauf
Spiel
Summary
A
B
A
It is not physics, but the understanding,
forecasting and steering of complex systems,
with the tools of a physicist.
It is
i (after many years of being a physicist at least)
equally fascinating.
What is absolutely great: our influence is larger!
First years were hard and risky: you have to sell!
There is no public money just coming in …
But with many very successful projects we now
have
a e many
a y contacts
co tacts to high-ranked
g a ed people
peop e
convinced of our capabilities.
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 200850
Start
Idee
NeuroBayes
Idee
Hintergrund
Ziele
NeuroBayes
f(t|x)
Beispiele
Historie
Anwendung
Prinzip
Funktion
Beispiel
Konkurrenz
Projekt l
Forschung
Projekt ll
Ablauf
Spiel
Summary
A
B
A
In general:
HEP community very effective and
well-trained Standard and motivation much
well-trained.
higher than in industry.
Phi-T, NeuroBayes® and its industrial
applications
pp
are a real spin-off
p
from High
g
Energy Physics with its origin in DELPHI.
btw: There is much to do!
Fascinating and challenging projects and permanent
positions available. Phi-T will grow!
From DELPHI to Phi-T
Michael Feindt
DELPHI Closing Meeting, CERN
May 200851