Hohmann M.R., Fomina T., Jayaram V., Widmann N., Förster C

A Cognitive Brain-Computer Interface for Patients
with Amyotrophic Lateral Sclerosis
Matthias R. Hohmann∗† , Tatiana Fomina∗† , Vinay Jayaram∗† , Natalie Widmann∗ , Christian Förster∗ ,
Jennifer Müller vom Hagen‡ , Matthis Synofzik‡ , Bernhard Schölkopf∗ , Ludger Schöls‡ , and Moritz Grosse-Wentrup∗
∗
Max Planck Institute for Intelligent Systems, Department for Empirical Inference, 72076 Tübingen, Germany.
Max Planck Research School for Cognitive and Systems Neuroscience, 72074 Tübingen, Germany.
‡ Hertie Institute for Clinical Brain Research, Department for Clinical Neurogenetics, 72076, Tübingen, Germany.
Email: {mhohmann, tfomina, vjayaram, nwidmann, bs, moritzgw}@tuebingen.mpg.de, [email protected],
{jennifer.mueller-vom-hagen, matthis.synofzik, ludger.schoels}@uni-tuebingen.de
† International
Abstract—Brain-computer interfaces (BCIs) are often based on
the control of sensorimotor processes, yet sensorimotor processes
are impaired in patients suffering from amyotrophic lateral
sclerosis (ALS). We devised a new paradigm that targets higherlevel cognitive processes to transmit information from the user
to the BCI. We instructed five ALS patients and eleven healthy
subjects to either activate self-referential memories or to focus
on processes without mnemonic content, while recording a highdensity electroencephalogram (EEG). Both tasks are likely to
modulate activity in the default mode network (DMN) without
involving sensorimotor pathways. We find that the two tasks can
be distinguished from bandpower modulations in the theta- (3–7
Hz) and alpha-range (8–13 Hz) in fronto-parietal areas, consistent
with modulation of neural activity in primary nodes of the DMN.
Training a support vector machine (SVM) to discriminate the two
tasks on theta- and alpha-power in the precuneus, as estimated
by a beamforming procedure, resulted in above chance-level
decoding accuracy after only one experimental session. Therefore,
the presented work could serve as a basis for a novel tool which
allows for simple, reliable communication with patients in late
stages of ALS.
Index Terms—EEG; brain-computer interface; brain-machine
interface; ALS; locked-in.
I. I NTRODUCTION
The usage of brain-computer interfaces (BCIs) in patients
suffering from late-stage amyotrophic lateral sclerosis (ALS)
holds the promise enabling communication. This mission has
proven to be very challenging in the final stages of the disease,
as BCIs are often based on motor- and sensory processes, such
as the volitional modulation of sensorimotor rhythms [1], [2].
However, patients suffering from ALS show degeneration of
neurons in the primary motor cortex [3], and impairment in
their ability to modulate these rhythms in later stages of the
disease. Visual speller systems require subjects to fixate on
target stimuli, which also fails due to impaired oculomotor
control [4]. For the same reason, patients are unable to make
use of the covert attention paradigms which rely on fixating on a
spot [5]. Tactile [6] and auditory [7] BCIs have only been tested
on healthy subjects and patients in earlier stages of the disease.
Their usefulness for establishing a reliable communication with
completely locked-in patients remains unclear. Therefore, we
c
978-1-4799-8697-2/15/$31.00 2015
European Union
propose a novel approach that incorporates non-sensorimotorrelated higher level brain functions that may be spared in ALS.
Higher-level brain functions can be incorporated into BCIs
by training subjects via neurofeedback to self-regulate neural
activity in cortical areas that subserve higher functions [8]. One
major issue with this approach is the amount of training that
is needed to modulate activity above chance-level. The need
for extensive training decreases the feasibility of the system,
especially for patients in later stages of the disease.
Recent studies have discussed mental tasks as an alternative
to motor imagery to improve BCI performance for disabled
users [9]. Here, we propose a novel cognitive strategy to
facilitate the manipulation of activity in the default mode
network (DMN), a large-scale cortical network that is involved
in self-referential processing [10] and is connected to the degree
of consciousness [11]. Based on these properties of the DMN,
we instructed subjects to alternate between self-referential
thoughts, which activate the DMN [12], and focusing on their
breathing, which deactivates the DMN because it is devoid of
self-referential mnemonics [13]. The current study investigates
the hypothesis that this strategy elicits bandpower changes in
the electroencephalogram (EEG) over areas consistent with the
DMN which are sufficiently strong to enable above chancelevel decoding accuracies in healthy subjects and patients with
ALS, without any subject training.
II. METHODS
A. Experimental Paradigm
Healthy subjects were placed in a chair approximately 1.25m
away from a 17” LCD screen with a resolution of 1280x1024
pixels and a 60 Hz refresh rate. The background of the screen
was black, with a white fixation cross appearing in the centre.
Prior to the experimental session, two five-minute resting state
EEGs were recorded. Subjects were asked to let their mind
wander and to either keep their eyes open in the first restingstate and closed in the second one. After the resting-state
sessions, subjects performed three experimental blocks with
brief intermissions. Each experimental block consisted of ten
trials in which the participants were asked to ”remember a
positive experience” and ten trials in which the participants
were asked to ”focus on their breathing”, in pseudo-randomized
order. Each trial began with 5.5±0.50 seconds rest, followed
by instructions that were given acoustically and visually to
indicate which of the two cognitive tasks should be performed.
After 60 seconds, the trial ended and the next trial started. To
ensure comprehension, both cognitive tasks were explained to
participants in a briefing before the experiment.
For ALS patients, the experimental paradigm remained
the same;however, they were only asked to perform one
experimental block.
B. Experimental Data
The study was conducted at the Max Planck Institute for
Intelligent Systems in Tübingen, Germany. Eleven healthy
subjects (eight male and three female, mean age 29.3 ± 8.3
years) and five ALS patients (cf. Table I) were recruited from
the local community and in cooperation with the University
Clinics Tübingen. Participants received 12 Euro per hour for
their participation. One healthy subject was excluded due to
noisy recordings. Another healthy subject and patient ET were
excluded due to technical disturbances during the experiment,
resulting in an unequal amount of trials per condition. This left
nine healthy subjects and four patients for the final analysis.
All participants were naive to the setup. They were informed
by the experimenter about the procedure and signed a consent
form to confirm their voluntary participation in advance. The
study was approved by the ethics committee of the Max Planck
Society.
TABLE I
ALS PATIENT DATA
Patient
GN
GV
HR
ET
LEK
1 Revised
Age
54
75
81
51
59
Sex
M
M
M
F
F
ALSFRS-R1
48
42
23
12
0
Impairment
Mild limb impairments
Mild limb impairments
No limb functionality
Locked-in, eye-movements
Residual eye-movements
1) Preprocessing: To compare healthy subjects and patients
with the same amount of trials, we focused on the first block
in our main analysis. We restricted our analysis to the timewindow of 6 to 60 seconds per trial, as instructions were given
in the first few seconds of each trial. To capture the effect
of self-referential processing, we restricted our analysis to
the α and θ-frequency bands (3Hz∼16Hz, lower and upper
α limit individually adjusted for each subject based on the
resting-state data, the lower θ limit was fixed at 3Hz), as selfreferential processing correlates with θ and α spectral power
[16]. The individual α-band for each subject was determined
by overlapping the spectral power of eyes open and eyes
closed resting states [17]. The lower and upper boundary of the
individual alpha band was set at the intersection point of the
two curves corresponding to the power spectra of the resting
states.
2) Attenuation of EMG artifacts: EEG recordings are likely
to be contaminated by scalp-muscle artifacts [18]. Therefore,
subjects may have been able to involuntarily influence the EEG
signal by altering the tonus of their scalp muscles. In order
to identify such EMG confounds, we employed independent
component analysis (ICA) [19]. The continuous data of one
session was first reduced to 64 components by principal
component analysis, and then separated into independent
components using the SOBI algorithm [20]. We then sorted
the ICs according to their neurophysiological plausibility [21],
manually inspected the topography, spectrum, and time-series
of each component, and rejected those for which at least one
of the following criteria applied: (1) Components displayed a
monotonic increase in spectral power starting around 20 Hz.
This is characteristic for muscle activity. (2) Eye-blinks were
detectable in the time series. (3) The topography did not show a
dipolar pattern. (4) The time series seemed to be contaminated
by other sources of noise, like bad impedance, large spikes, and
50 Hz line noise. As discussed in [22], it is unreasonable to
expect a complete removal of artifacts using ICA, but careful
application is a useful means of rejecting the most dubious
results on the scalp.
amyotrophic lateral sclerosis functional rating scale [14]
A 124-channel EEG was recorded at a sampling frequency
of 500 Hz using actiCAP active electrodes and a QuickAmp
amplifier (provided by BrainProducts GmbH, Gilching, Germany). Electrodes were placed according to the extended 10-20
system with the left mastoid electrode as the initial reference.
All recordings were converted to common average reference.
The application was realised with the BCI2000 and BCPy2000
toolboxes [15].
C. Data Analysis
We performed an offline analysis on the acquired data to
identify confounding EMG activity, to analyse the cortical distribution of the induced effect, and to investigate differentiability
of the activity-patterns associated with self-referential thoughts
and focus on breathing.
Fig. 1. Topography of sources that represent the precuneus targeted by the
beamforming procedure.
3) Pattern Classification: Because the precuneus is a hub
of the DMN [23], we aimed a linearly constrained minimum
variance (LCMV) beamforming algorithm [24] at this region.
First, the subject-specific covariance matrix Σ ∈ RN ×N of
the resting state recording with N = 124 EEG channels was
computed. Then, the beamformer was computed by solving the
LCMV-optimization problem
Fig. 2.
Fig. 3.
Average cortical R2 -maps for θ and α-bandpower across all patients.
Average cortical R2 -maps for θ and α-bandpower across all healthy subjects.
w∗ = argmin wT Σw s.t. wT a = 1
with a ∈ RN being the scalp topography based on a subset
of sources in a forward model that represents the precuneus
(cf. Figure 1 and [8]). The resulting spatial filter w∗ was then
applied to the EEG data x [t] ∈ RN to obtain a 1D signal
y [t] = w∗T x [t] from the target area, in which the variance of
all sources outside this area is optimally attenuated. For each
trial, we windowed the resulting current density estimate in the
precuneus with a Hann window, performed a DFT and averaged
over the combined θ- and α-range to get a log-bandpower
estimate for the trial. This served as our one-dimensional feature
space.
We then employed a support vector machine (SVM) with
a linear kernel to estimate the accuracy in discriminating the
activity-patterns of both tasks based on the combined θ and α
bandpower as described above. We employed a leave-one-trialout cross-validation procedure within each subject. The optimal
regularisation parameter for each training-set was determined in
an inner loop and afterwards used in an outer loop to determine
the classification accuracy. To test the null-hypothesis H0 :
median classification-accuracy = 0.5 (chance-level), we used
a Wilcoxon signed-rank test, given the classification-accuracy
value of each subject.
4) Spatial Distribution: To further investigate the spatial
distribution of the effect, we computed the coefficient of
determination (R2 ) for every subject and channel to evaluate
the average difference in band-power in α and θ bands induced
by the two tasks.
III. RESULTS
Table II and III show the accuracy of the classification of the
beamformed, combined α and θ-bandpower as described above
for the ALS patients and healthy subjects, respectively. A onetailed Wilcoxon signed-rank test rejected the null-hypothesis
of a median classification accuracy on chance-level (50%) at
p < 0.001 for the combined subject groups.
Figures 2 and 3 show the R2 -values on a topographic plot
for the combined α and θ-bandpower in patients and healthy
subjects. A higher R2 -value depicts a higher percentage of
variance explained by the experimental conditions. It can be
seen that the observed modulation centres around the parietal
as well as the prefrontal cortex, especially in the α-band.
TABLE II
C LASSIFICATION ACCURACY FOR PATIENTS
GV
70%
LEK
65%
GN
75%
HR
65%
TABLE III
C LASSIFICATION ACCURACY FOR H EALTHY S UBJECTS
S1
85%
S2
45%
S3
80%
S4
90%
S5
80%
S6
55%
S7
70%
S8
55%
S9
70%
IV. DISCUSSION
The current study aimed to show that the employed cognitive
strategy modulates activity in the precuneus, and that this
modulation can be differentiated above chance level without
any subject training. Using a linear kernel SVM, we were able
to successfully classify both patterns with an average decoding
of 70% and 68.75% for the healthy subjects and ALS patients
respectively. We further investigated the spatial distribution of
the effect by looking at R2 values for each EEG channel in the
θ- and α-range. Larger R2 values can mostly be seen around
parietal and prefrontal areas, consistent with modulation of
neural activity in primary nodes of the DMN. Additionally,
α and θ-bands show slightly different spatial patterns, which
may indicate that both bands are involved in different cognitive
processes. Most importantly, we found all four ALS patients
in various stages of disease progression to be capable of selfmodulating activity in the targeted areas without extensive
training.
However, one has to be careful when interpreting these
results in terms of communication. All data was analysed offline
employing a cross-validation as a way of obtaining predictionerrors that are nearly unbiased [25]. This method cannot be
employed during online use. Online, a classification accuracy
≥ 70% is usually considered necessary for communication.
Similarly, according to Müller-Putz et al. [26], a classification
accuracy of 70% is needed for 10 trials and two classes to
reject the null-hypothesis in an online setting with p < 0.05.
Additionally, 20 trials per participant limit the interpretability of
individual classification results. Future research could therefore
investigate the use of different decoding models, such as a
multitask learning approach [27]. This method may allow for
offline training on multiple subjects in multiple sessions, and
could then be employed for online classification. Due to a
larger set of training data, it may also allow for a comparison
with unrestricted feature derivation on sensor-level.
While subjects effectively modulated activity in the precuneus, we found a large variability in bandpower differences
on an individual level. One reason for this could be the choice
of the non-self-referential condition. Focusing on breathing has
been shown to decrease overall activity in the DMN, but it also
increases synchronisation within the DMN [28]. These two
effects may be difficult to separate when investigating EEG
bandpower-values, as an increase in synchronisation can lead
to an increase in spectral-power, indistinguishable from the selfreferential activation. A potential solution to this problem could
be the choice of a different non-self-referential strategy. One
candidate could be a verbal spelling task, as verbal execution
has been found to lower DMN activity [29].
The successful implementation of this novel cognitive
strategy has a number of implications for further development
of BCI systems for ALS patients. First, recordings were
conducted with an 124-channel wet-electrode EEG system.
Such conventional EEG systems are often only accessible
in clinical environments. They are not very cost-efficient or
portable. Also, nursing staff or family members of the ALS
patient may not have the necessary expertise to setup such a
conventional EEG system for online-communication. To create a
communication method that is available to everyone, it would be
beneficial to transfer the paradigm to a commercially available,
less expensive, and portable EEG system. As the effects were
very pronounced with this strategy, it could help to stabilise
the effect in a way that it can still reliably be detected with
such a low-density, low-cost system. Second, extensive training
can be very exhausting for the patients, especially during later
stages of the disease. Therefore, the training time needs to be
reduced to make the system more applicable. Our strategy may
achieve this by providing initial guidance for the use of the
system. Last, the strategy is relatively easy to understand and to
employ which may help to further simplify the task and make
it more appealing. This would open opportunities for easily
conductible studies. Most importantly, it could serve as a basis
for an easy-to-use novel tool which allows simple, reliable
communication with completely locked-in ALS patients.
ACKNOWLEDGMENT
We would like to thank Bernd Battes for assistance with the
technical equipment.
R EFERENCES
[1] G. Pfurtscheller and C. Neuper, “Motor imagery and direct brain-computer
communication,” Proceedings of the IEEE, vol. 89, no. 7, pp. 1123–1134,
2001.
[2] A. Kübler, F. Nijboer, J. Mellinger, T. M. Vaughan, H. Pawelzik, G. Schalk,
D. J. McFarland, N. Birbaumer, and J. R. Wolpaw, “Patients with ALS
can use sensorimotor rhythms to operate a brain-computer interface,”
Neurology, vol. 64, pp. 1775–7, May 2005.
[3] K. Nihei, A. C. McKee, and N. W. Kowall, “Patterns of neuronal
degeneration in the motor cortex of amyotrophic lateral sclerosis patients,”
Acta Neuropathologica, vol. 86, pp. 55–64, June 1993.
[4] L. Jacobs, D. Bozian, R. R. Heffner, and S. a. Barron, “An eye movement
disorder in amyotrophic lateral sclerosis,” Neurology, vol. 31, pp. 1282–
1287, Oct. 1981.
[5] M. Treder, N. Schmidt, and B. Blankertz, “Gaze-independent braincomputer interfaces based on covert attention and feature attention,”
Journal of Neural Engineering, vol. 8, p. 066003, Dec. 2011.
[6] A. M. Brouwer and J. B. F. van Erp, “A tactile P300 brain-computer
interface,” Frontiers in Neuroscience, vol. 4, p. 19, Jan. 2010.
[7] F. Nijboer, A. Furdea, I. Gunst, J. Mellinger, D. J. McFarland, N. Birbaumer, and A. Kübler, “An auditory brain-computer interface (BCI),”
Journal of Neuroscience Methods, vol. 167, pp. 43–50, Jan. 2008.
[8] M. Grosse-Wentrup and B. Schölkopf, “A brain-computer interface based
on self-regulation of gamma-oscillations in the superior parietal cortex,”
Journal of Neural Engineering, vol. 11, no. 5, p. 056015, 2014.
[9] R. Scherer, J. Faller, E. V. C. Friedrich, E. Opisso, U. Costa, A. Kübler,
and G. R. Müller-Putz, “Individually adapted imagery improves braincomputer interface performance in end-users with disability,” PloS ONE,
vol. 10, p. e0123727, Jan. 2015.
[10] R. N. Spreng, E. DuPre, D. Selarka, J. Garcia, S. Gojkovic, J. Mildner,
W.-M. Luh, and G. R. Turner, “Goal-congruent default network activity
facilitates cognitive control,” Journal of Neuroscience, vol. 34, no. 42,
pp. 14108–14114, 2014.
[11] A. Vanhaudenhuyse, Q. Noirhomme, L. J. F. Tshibanda, M. A. Bruno,
P. Boveroux, C. Schnakers, A. Soddu, V. Perlbarg, D. Ledoux, J. F.
Brichant, G. Moonen, P. Maquet, M. D. Greicius, S. Laureys, and M. Boly,
“Default network connectivity reflects the level of consciousness in noncommunicative brain-damaged patients,” Brain, vol. 133, pp. 161–171,
Jan. 2010.
[12] N. C. Andreasen, D. S. O’Leary, T. Cizadlo, S. Arndt, K. Rezai, G. L.
Watkins, L. L. B. Ponto, and R. D. Hichwa, “Remembering the past: Two
facets of episodic memory explored with positron emission tomography,”
American Journal of Psychiatry, vol. 152, pp. 1576–1585, Nov. 1995.
[13] M. E. Raichle, a. M. MacLeod, a. Z. Snyder, W. J. Powers, D. a. Gusnard,
and G. L. Shulman, “A default mode of brain function,” Proceedings
of the National Academy of Sciences of the United States of America,
vol. 98, pp. 676–682, Jan. 2001.
[14] J. M. Cedarbaum, N. Stambler, E. Malta, C. Fuller, D. Hilt, B. Thurmond,
and A. Nakanishi, “The ALSFRS-R: A revised ALS functional rating
scale that incorporates assessments of respiratory function,” Journal of
the Neurological Sciences, vol. 169, no. 1-2, pp. 13–21, 1999.
[15] G. Schalk, D. J. McFarland, T. Hinterberger, N. Birbaumer, and J. R.
Wolpaw, “BCI2000: A general-purpose brain-computer interface (BCI)
system,” IEEE Transactions on Biomedical Engineering, vol. 51, pp. 1034–
1043, June 2004.
[16] Y. Mu and S. Han, “Neural oscillations involved in self-referential
processing,” NeuroImage, vol. 53, pp. 757–768, Nov. 2010.
[17] W. Klimesch, “EEG alpha and theta oscillations reflect cognitive and
memory performance: A review and analysis,” Brain Research Reviews,
vol. 29, pp. 169–195, Apr. 1999.
[18] A. J. Shackman, B. W. McMenamin, H. a. Slagter, J. S. Maxwell,
L. L. Greischar, and R. J. Davidson, “Electromyogenic artifacts and
electroencephalographic inferences,” Brain Topography, vol. 22, pp. 7–
12, June 2009.
[19] B. W. McMenamin, A. J. Shackman, J. S. Maxwell, D. R. W. Bachhuber,
A. M. Koppenhaver, L. L. Greischar, and R. J. Davidson, “Validation of
ICA-based myogenic artifact correction for scalp and source-localized
EEG,” NeuroImage, vol. 49, pp. 2416–2432, Feb. 2010.
[20] A. Belouchrani, K. Abed-Meraim, J.-F. Cardoso, and E. Moulines, “A
blind source separation technique using second-order statistics,” IEEE
Transactions on Signal Processing, vol. 45, no. 2, pp. 434–444, 1997.
[21] M. Grosse-Wentrup, S. Harmeling, T. Zander, J. Hill, and B. Scholkopf,
“How to test the quality of reconstructed sources in independent
component analysis (ICA) of EEG/MEG data,” in Proceedings - 2013
3rd International Workshop on Pattern Recognition in Neuroimaging,
PRNI 2013, pp. 102–105, 2013.
[22] B. W. McMenamin, A. J. Shackman, L. L. Greischar, and R. J. Davidson, “Electromyogenic artifacts and electroencephalographic inferences
revisited,” NeuroImage, vol. 54, pp. 4–9, Jan. 2011.
[23] P. Fransson and G. Marrelec, “The precuneus/posterior cingulate cortex
plays a pivotal role in the default mode network: Evidence from a partial
correlation network analysis,” NeuroImage, vol. 42, no. 3, pp. 1178–1184,
2008.
[24] B. D. Van Veen, W. Van Drongelen, M. Yuchtman, and A. Suzuki,
“Localization of brain electrical activity via linearly constrained minimum
variance spatial filtering,” IEEE Transactions on Biomedical Engineering,
vol. 44, pp. 867–880, Sept. 1997.
[25] B. Efron and G. Gong, “A leisurely look at the bootstrap, the jackknife,
and cross-validation,” The American Statistician, vol. 37, pp. 36–48, Feb.
1983.
[26] G. Mueller-Putz, R. Scherer, C. Brunner, R. Leeb, and G. Pfurtscheller,
“Better than random: A closer look on BCI results,” International Journal
of Bioelectromagnetism, vol. 10, no. 1, pp. 52–55, 2008.
[27] M. Alamgir, M. Grosse-Wentrup, and Y. Altun, “Multitask learning for
brain-computer interfaces,” in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS-10),
(Sardinia, Italy), pp. 17–24, 2010.
[28] J. a. Brewer, P. D. Worhunsky, J. R. Gray, Y.-Y. Tang, J. Weber, and
H. Kober, “Meditation experience is associated with differences in default
mode network activity and connectivity,” Proceedings of the National
Academy of Sciences, vol. 108, pp. 20254–20259, Dec. 2011.
[29] H. Koshino, T. Minamoto, K. Yaoi, M. Osaka, and N. Osaka, “Coactivation of the default mode network regions and working memory network
regions during task preparation,” Scientific Reports, vol. 4, p. 5954, Jan.
2014.