- Welcome everyone to Wednesday Nite @ The Lab .
I'm Tom Zinnen, I work here at
the UW-Madison Biotechnology Center.
I also work for UW Extension- Cooperative Extension,
and, on behalf of those folks and our other co-organizers,
Wisconsin Public Television,
the Wisconsin Alumni Association,
and the UW-Madison Science Alliance, thanks again
for coming to Wednesday Nite @ the Lab .
We do this every Wednesday night, 50 times a year.
Tonight, it's my pleasure to get to introduce to you
Professor Ruth Litovsky.
She's a professor in the Department
of Communication Sciences and Disorders and also in Surgery.
She was born in Israel, grew up in Chicago,
got her undergraduate degree at Washington University
in St. Louis, then came here for a PhD,
went to Boston University and MIT for post docs,
came back here to be on the faculty.
Just last year, she was a Fulbright fellow
in Melbourne, Australia.
She's gonna to be talking with us about one of
the more remarkable technologies I've heard about,
that's the cochlear implant,
otherwise known as the bionic ear.
Please join me in welcoming Professor Litovsky
to Wednesday Nite @ the Lab .
(audience applauds)
- Thank you so much, Tom, for the invitation.
Good evening, everybody.
It's my pleasure to be here tonight to speak to you
about research that we've been doing in my lab here
at the Waisman Center for
the past 14, 15 years or so.
We call it the bionic ear because one might wonder
whether, with electrical stimulation, you might hear better
than you would with acoustic hearing.
And it's true for people who have severe
or profound hearing loss.
So I'd like to start by talking about the human ear.
And the human ear is remarkable.
I'm gonna show you this in a sort of 3-D rendition,
which my students love to see
because you're looking through the external ear
into the middle ear.
And then, if I freeze this for a second, you see
this lovely, wide, blue band, which
is your eighth nerve, cochlear nerve, or auditory nerve.
This is the nerve that we stimulate, if you have
hearing loss in the cochlea, if the hair cells have died.
I'll show you pictures of what damage looks like
in the cochlea, but the idea is that we position an array
of electrodes into this swirl, the cochlea.
We stimulate the auditory nerve, and then the brain has to
take care of a lot of what goes on next
with interpreting the sound.
So this is a picture of the cochlea and its bony labyrinth.
It's a beautiful snail-shaped structure
only 35 millimeters in length in an adult.
It is mature at birth, though, which is quite remarkable,
which is why we can implant infants
when they're quite young.
The issue in infants is are they hefty enough,
in terms of blood loss, to sustain the surgical procedure?
But otherwise, the cochlea can take an implant
at a very, very young age.
The cochlea is also remarkable
because we have a map of frequency.
We call it tonotopic organization.
So from the base of the cochlea, as we move
up to what we call the apex, as we move
through 2 1/2 turns in the human ear,
we go from very high frequencies,
most of us don't really hear about 20,000 hertz,
but we move on a logarithmic scale
as we go through to the apex.
And this wavy bit here is what we call the traveling wave,
which is that stimulation travels from the base
toward the apex, and the cochlea is a frequency analyzer.
So every place along the cochlea resonates best
with a given frequency.
And so as the sounds travel through the cochlea,
they find their sweet spot, or their best place
at which that frequency best stimulates
that basilar membrane, or the membrane on the cochlea
that responds to sound.
So we have maybe one place along the cochlea
that responds best to one frequency, another place
to another frequency, et cetera.
This is what a little video of
the basilar membrane looks like.
This is the membrane in which the hair cells sit.
And so as the traveling wave comes through the cochlea,
this membrane will vibrate in an oscillatory motion
up and down, in response to sound.
When there is no sound, it doesn't vibrate.
When the sound finishes, it stops to vibrate.
But this vibration occurs at all places along the cochlea.
And this is a very important part of acoustic hearing.
We bypass all of this in the bionic ear.
So it's quite remarkable that we can lose
what we call cochlear mechanics and still stimulate
the auditory nerve and have the brain interpret sound.
Speaking of cochlear mechanics, this is a little video
of what a hair cell looks like.
And hair cells have to move in order,
these are outer hair cells,
they have to move in order for the basilar membrane
to amplify sound.
So if you're somebody with hearing loss
of about 50 decibels,
moderate toward severe hearing loss, this is
because the outer hair cells have ceased to function,
or they have been damaged, and they cannot move.
And the loss of mechanical movement leads
to loss of amplification or loss of hearing
and other limitations that I can address as well later on.
But let me play for you this really cute video.
It's courtesy of Professor Ashmore.
And you'll recognize the song, and you'll see
the hair cell vibrating at the percussion of the song.
? One, two, 3 o'clock, 4 o'clock rock
? Five, six, 7 o'clock, 8 o'clock rock
? Nine, ten, 11 o'clock, 12 o'clock rock
? We're gonna rock around the clock tonight
? Put your glad rags on and join me hon
? We'll have some fun when ?
Okay, you get the idea. (laughs with audience)
But it's a really nice way to show cochlear mechanics,
which is that there's physical vibratory movement
of these hair cells and loss of cochlear mechanics
is hugely responsible for loss of hearing.
So seen here
a cross-section of the cochlea
and the hair cells, we have outer and inner hair cells.
It is these guys, the inner hair cells, that actually
transmit information up to the brain.
So we have the cochlear mechanics, but then we have these
inner hair cells.
Their loss or their death is what primarily causes
what we call severe to profound hearing loss,
and lack of ability to really benefit
from hearing aids in acoustic hearing.
So we have these beautiful rows of hair cells,
and I'll talk a little bit about what they look like
when they're damaged, but the mechanics,
also if you look down here at the bottom,
involve the physical movement
of these teeny little hair cells to open ion channels
that lead to transduction of ions into the cell
and hearing as well.
I love to talk about this because the cascade
of events is beautiful, it's complicated,
and at every place along the way, the loss of any
of these pieces can lead to hearing loss.
So, information about sound is transmitted to our brain
through networks of neurons.
I've talked about the periphery up until now,
but let's not forget about the brain.
And so here, we're looking at the cochlea
and the tonotopic organization, the map
of frequency that I described to you,
but all of this is transmitted up to higher levels
in the brain, and at each step along the way,
we have a new level of analysis of the signal
of the information, which, again
is what's really important to lead to hearing.
So we have the frequency map.
That is maintained as we move up all the way
to the auditory cortex.
Lower down in the system, we have the ability
to resolve spectral information, or resolve
the components of speech cues.
As we move a little bit higher up, we have the ability
to combine information across the two ears
so that we can localize sound,
and so we can separate speech in noisy environments.
And then, of course, let's not forget
that hearing involves listening,
so we've got attention, memory, emotion,
and I'll talk a little bit later, toward the end,
about cognitive load or listening effort
and the difficulty that people experience
in noisy environments and how we're trying to study that.
So there's a very important concept here,
which is top down processing.
The signal travels up the pathway,
but the brain has to enact on the signal
in order for us to interpret it.
I would like to give you an example
of what top-down processing and online recognition
of signals might sound like.
So I'm going to play for you a degraded form of speech.
We call it sine-wave speech.
And the first thing I'll do is play a sound
that I don't think you'll be able to understand.
(voice speaks distorted, machine-like speech)
Anybody understand that?
Okay good.
(voice speaks distorted, machine-like speech)
Now here's the clear speech version of that.
- [Voiceover] She cuts with her knife.
- [Ruth] Now listen again to the degraded speech.
(voice speaks distorted, machine-like speech)
(audience murmurs) Aah.
So you have a filling in of the missing information
because you know what you're listening for.
I'll play one more just 'cos they're fun.
This is my colleague Quentin Sommerfield from the UK.
(voice speaks distorted, machine-like speech)
Again.
(voice speaks distorted, machine-like speech)
- [Voiceover] They're buying some bread.
- [Ruth] Did you get it? Oh, very nice.
So you're learning to use the sine-wave speech.
(voice speaks distorted, machine-like speech)
So what you're getting is this pop-out effect
or insight into the signal.
This is a little bit like what people
who use cochlear implants have to do.
They have a really degraded signal.
I'll play some more examples for you.
They're having to fill in the information
based on top-down processing, based on learning and training
and knowing how to interpret the signal.
So what we're dealing with
in terms of a bionic ear in situation is,
sensori-neural hearing loss,
hearing loss in the periphery.
We're dealing with permanent types of hearing loss.
They're not restorable today with stem cells
or hair-cell regeneration
or some of the other beautiful technologies
that we would love to be able to have
in this population.
We're dealing with a comparison of normal hair cells,
and this is what damaged hair cells look like.
So they can't move,
they can't have ion channels open.
They're basically not functional.
And there are a lot of issues that I can talk about
a little bit later if we have time,
which are the education and social issues in particular
for children with hearing loss.
Do you mainstream them?
How do you mainstream them in educational situations
if they have hearing loss, if they're struggling to listen?
What about school for the deaf?
That's been a very interesting issue
of the deaf culture with a capital D.
So I won't really talk about those as much as
the science behind the bionic ear,
but I'm very sensitive to these issues.
So, we can have progressive hearing loss,
which is that many adults, and also some children
who are born with normal hearing,
can experience hearing loss as it progresses
over a period of months or years, until eventually,
they become profoundly deaf.
There are many, many causes.
I'm only listing a few here.
Meningitis is an example of sudden hearing loss.
Accident is an example of sudden hearing loss.
Oto-toxicity, you can accidentally end up being given
a cocktail of drugs to deal with other medical issues,
but they are toxic to the ear, and they can cause deafness.
But noise exposure and heredity
are examples of loss of hearing
that are often progressive.
They can take months and years to get to where they are,
and often, there are, of course, unknown reasons.
And again, on the right, you see examples
of damaged, dead, lost hair cells,
compared on the left with healthy hair cells.
So, when we think about what cochlear implants help us with,
we're thinking about the importance of communication
and learning in natural, everyday listening environments.
This is where we want people to be able
to participate, to succeed,
to learn, to grow, grow,
and to be members of society.
Now, a lot of the issues that we deal with are
whether you can know where sounds are coming from
so that you can direct your attention to them.
And a little bit later, I'll talk about the fact
that this depends a lot on whether you can have access
to sound in both ears.
So, cochlear implants were designed
to give input to one ear.
Later on, we started to see a progression and a push
in the field to give people two cochlear implants.
And one of the things that having two cochlear implants
helps you with is improvement in localizing sounds,
in knowing where to direct your attention.
We also expect children to acquire language
in complex environments like classrooms
and to be able to listen to what we call streamed speech
and to predict what is about to come next, and
to follow instructions.
And this is very complex.
And being able to also segregate
the important speech sounds from noise in the environment
is part of what they have to do to engage in the real world.
So they have to hear a target in a noisy environment.
They have to segregate speech from noise
or from other distracting sounds.
Cochlear implants in one ear don't do
a terribly good job of that.
So again, having input in both ears makes
a very big difference.
We want children and infants to be able to do everything
that I have just talked about without being overwhelmed.
And I like to point this out because giving somebody
a cochlear implant surgically is one step.
It's the very first step.
There are many steps along the way in which we have
to program the device and make sure
that they can use it well.
They have to revisit the audiologist,
and the device has to work well.
And if it doesn't work well, if it's set to be too loud,
they can be over-whelmed, they might choose
to simply take it off their ears.
So you really have to work very closely
between the parents, the school audiologist,
many, many other professionals
that are involved in treating these children.
We don't want them to get confused
about what they're hearing.
Part of the confusion comes from
not having enough speech therapy.
So habilitation, rehabilitation speech therapy
are all part of training children and adults
to learn their devices.
And of course, we want them to enjoy their auditory input.
And one thing that I want to point out right now is,
music is very tricky.
And I'll explain a little bit why.
But the speech signal and the music signal
that come in through the implants are somewhat degraded.
And so there's a little bit of a tricky issue relating
to music, and it's one of the areas that
the field is working on.
So what does the implant do?
You're looking at
a simulation of electrodes that are sitting in the cochlea.
They're stimulating the auditory nerve.
The goal is to bypass the damaged hair cells,
stimulate the nerve.
Today children receive these devices
definitely by 12 months of age,
but I speak to surgeons all over the world who,
I watched a surgery the other day in Germany,
where they implanted an infant simultaneously in both ears
at eight months of age.
So the standard of care has really shifted over the years.
It depends a lot on issues like what the parents want,
what the surgeon recommends, if a child is old enough
to say what they want, that gets taken into account,
and money.
Depends on whether the health insurance company
will cover the cost.
So adults who become deaf are also eligible,
but not in every country.
And so that's a very interesting part of the issue.
But today, we estimate there are over 350,000 people,
and the numbers are growing.
These are pictures of some of the kids
that we have studied in the lab,
and one of the things I want to mention
is, you can see here that they really also have
to work with the ergonomics of where are they gonna put
these devices so the child won't rip it off the head.
But here, you see microphones that are sitting
on the shoulders rather than behind the ear.
That's not ideal.
And so part of what I try to do when they come
to my lab for my research is talk about training both
the parents and the audiologists to use the devices
to the best way that is possible.
And this picture of this little girl is exciting to show
because she has two pigtails in this picture.
But when she came to my lab,
she had received two cochlear implants at a year of age.
She was now three, but she had both of them on a ponytail
on top of her head rather than behind each ear.
So she wasn't getting the spatial cues that we get
by being able to hear sounds on both sides of the head.
But talking to the parents 101 (laughs)
on cochlear implants and sound localization,
they were right there with me, they redid her hairdo,
she's wearing two devices with two pigtails.
Little things like that, which is sort of bench to clinic,
taking the information that we have in the lab
and really working with the population
to get the information out there.
So we have sound through electrical stimulation.
What we're seeing on this slide is the fact
that we're bypassing anything that the ear does,
the ear canal, the bones, which are really important,
and we're going straight into this swirly shape,
the cochlea, stimulating the auditory nerve.
In here you see,
there are only about 22 electrodes,
and this is very different than the tens of thousands
of hair cells and of auditory nerve fibers
that are responding separately and beautifully
to each frequency.
So we have, as I'll talk about in a minute,
a degradation of the speech signal.
At the bottom right, you see what is the actual implant
that goes into the bone, and so,
what happens is that it's implanted into the temporal bone,
and the information is transmitted through a coil
through an RF signal through the scalp.
This also means that when a child or an adult takes
the implant, the outside part off, they are deaf again.
So they can hear when they're being stimulated.
They take it off to swim, to have a shower,
mostly to go to sleep at night, then they are deaf.
And so this is a very interesting part
of coping with hearing most of the time
but not always hearing.
And if the battery dies, they become deaf.
And so they have to cope with those situations as well.
So what we are doing is we are trying to replace
acoustic, rich, beautiful acoustic stimulation
with electrical stimulation, and we are taking advantage
of that tonotopic organization,
the map of frequency in the cochlea.
So, recall that we have a map that goes
from the base, 20,000 Hertz,
to the apex,
where we have very low frequencies.
Here you see an electrode array going into the cochlea,
but it stops over here after about one turn.
We can't get them in too deep
because of the anatomy of the cochlea
and the virtual size, the actual size
of the electrode array.
So we're remapping the brain
to use frequencies
at places that the brain isn't used to getting.
There's a lot of adaptability and plasticity
that has to happen.
Because, for example, we send all the frequencies
to all of these electrodes.
This place in the cochlea
likes about 1,500 hertz,
but it's going to receive the lowest frequency possible,
maybe 200 hertz.
So, there's a shifting of frequency and adaptability
that the brain has to go through,
which is why, when you first turn on
a cochlear implant, it might sound bionic,
or fuzzy, or a little bit strange,
and it takes time to get to know how
to use the information.
Little bit of history here, very interesting history,
which is that the early invention was around 1979.
Then implants began to be provided
to children around 1985.
In the 1990s is when the discussion happened
about whether people should get implants in both ears
or not, and that's when clinical trials began
in the United States.
What's really interesting is to talk to some of
the engineers who were involved
in developing these devices in the '70s and '80s.
I happened to meet one of them
when I was in Australia last year, Peter Seligman,
and he says, "You know, when I started doing that work,
"we never believed it would ever work,
"but we thought we'd have a lot of fun trying."
(audience laughs)
And it's really amazing how far it has come,
and in fact, now, he and others are working
on retinal implants, and he has the same attitude.
"I don't think it'll work, maybe it'll work, maybe not,
"but we're having fun trying."
And so that's a true engineer, that's really
the true attitude of an engineer.
But it takes engineers,
otology physicians, audiologists,
speech pathologists, and researchers
to really put all of this together.
So around 2005 is when two cochlear implants
became more standard of care.
That's when health insurances really started
to cover the cost of the devices.
The other thing that happened is that implants started
to be provided also in other places, like
beyond the auditory nerve.
So there are disorders in which
the auditory nerve doesn't work.
So what do you do then?
Well, maybe you can bypass the auditory nerve
and stimulate the brain stem, the cochlear nucleus,
or the midbrain.
So smaller numbers of patients,
but definitely moving the field in that direction.
It also became clear that not everybody
has profound loss of hearing
all the way along the cochlea.
Many people have what we call residual, or leftover,
hearing in the low frequencies.
So we don't want to put the cochlear implant all the way in
and destroy whatever hair cells are still surviving there.
And so there are now what we call short electrode arrays
that try to preserve whatever hearing
people still have left.
We call that electric-acoustic hearing, which is that,
you might have a cochlear implant in your ear,
it's only halfway in, you can use a hearing aid combined
with a cochlear implant and have hybrid hearing,
or electric-acoustic hearing.
You might have one implant in one ear and a hearing aid
in the other ear and be able to use those together as well.
So there lots of advances in the field
we call the preservation of hearing.
We don't want to ruin what you still have left,
so we want to let you use what you have, preserve hearing,
and use the technology as well.
The other very interesting thing that started
really recently, a few years ago,
is what we call cochlear implantation in people
with single-sided deafness.
So somebody with
perfectly good hearing in one ear,
deafness in the other ear, often due to tinnitus.
So, people who have severe tinnitus in one ear
and, it's accompanied by deafness, are starting to receive
an implant in that ear.
Well, it turns out that for many of those patients,
the electrical stimulation suppresses the tinnitus
and leads to an enormous improvement in quality of life.
And it started out that way.
Once those floodgates began to open, other people
with deafness in one ear said, "I would like
"a cochlear implant, so that I can hear in both ears,"
and some of those have paid out of pocket, about $60,000,
but some of them have had insurance companies pay
for them as well.
So kind of a wave into the future that's very interesting.
So, for the engineers out there, I'm gonna take
less than a minute of your time to just mention
what we mean when we talk about signal processing.
We have a speech waveform here on the left.
We take that waveform, and we break it down.
We call it bandpass filtering.
So we filter that sound into those 20-some channels.
We then extract the envelope of that signal.
We have to do what's called compression in that signal.
We then have pulsatile stimulation that is sent to
all of the electrodes, and now you see
the auditory nerve firing.
But that yellow shape
is showing you that,
every time I stimulate any place along the cochlea,
I'm not very good at containing to just that place.
We have what's called a spread, which means that the signal
is not that clear.
And so we're not able to present stimulation
to only one place and keep it just there.
There's a spread of excitation.
And so that's part of why the signal sounds
a little bit degraded.
So we have some limitations,
some distortion in the level cues,
the two ears are not fitted in a way
that they're synchronized, so that's also a problem.
And we have this limited spectral resolution.
But we have a lot of happy results
or a lot of people who are very happy with their results.
I really like showing this picture in particular
on the right because this family has two parents
who are both deaf from birth.
They sign and they speak.
They then had two children who were deaf.
And within a year after being born,
each of those children had two cochlear implants
because the parents wanted the kids to sign but to speak.
And, I have to tell you, I've met these kids many times.
They come to my lab every year for research,
and they speak beautifully.
And they have many, many hours a week of speech therapy,
but they also sign with their parents.
They're bilingual.
The parents then each got a cochlear implant
because they wanted to know what it's like for the children
to have to use them.
So the parents don't have clear speech
or very clear understanding of the signal,
but they understand that sounds are happening in the world.
They use the sound to augment their speech understanding
and their lip-reading.
This guy over here on the left
is a graduate from UW-Madison.
He became deaf as a child, and by age 12 received
an implant, and he really loves
having just one in one ear.
He functions beautifully.
I can call him from the other side of the room,
and he hears me perfectly well.
He was a cheerleader here at UW, he was an athlete,
functions really, really well.
And so, you see, a lot variability in what people choose
and also in the outcomes, partly depending on the history
and partly depending on unknown factors.
I'd like to play some sounds for you
of what we think simulations
of cochlear implants sound like.
I'll play the sound initially with what we call
an 8 channel simulation of a girl who is talking.
And this is what it might sound like when
the activation of the implant first happens.
See if you can understand the speech.
(voice speaks distorted, metallic speech)
Okay, let's make it a little bit clearer with more channels.
(voice speaks distorted, metallic speech)
- [Ruth] More channels.
- [Voiceover] Wanna come to my birthday party in August?
- [Ruth] And then actual speech.
- [Voiceover] Wanna come to my birthday party in August?
(voice speaks distorted, metallic speech)
- [Ruth] Now that you know what you're listening for,
it's much easier to interpret the signal,
and cochlear implant users say
that their improvement happened very quickly over months,
and by a year, they really stabilized quite well.
Another example that I have is of music.
So let's listen to the 8-channel music.
Uh oh. Sorry.
I'm not sure what's happening there.
I think my music demo's not working.
So, sorry about that.
I can put it online.
So here's some important considerations.
The age at implantation, which I have mentioned.
Now we're going down to very, very young age,
but that's not available in every country.
The quality of the signal that is reaching the brain.
The training.
Training the users to interpret the signal
so that speech and language can develop.
The brain has to interpret, define, use memory and cognition
to interpret the signal to fill in what is going on.
So I really can't emphasize enough the importance
of habilitation or rehabilitation.
So how well do children
with cochlear implants acquire language?
These are data from a study that was published
a number of years ago.
On the left, we see comprehension, which is
the ability to understand language,
as a function of the test age in years,
and their scores.
What we see here are the normal hearing children.
As they get older, they ramp up very quickly.
And then we see three lines.
The yellow are children who received their implants
before 18 months of age.
The blue are children who received them
between 18 months and three years of age,
and the green are children who waited
more than three years of age.
And you can see that the age of implantation has
a huge effect on the ability of children
to ramp up their language comprehension
on standardized measures of language.
On the right, we see similar findings for expression,
so testing the ability of a child to express themselves
on standardized measure of language.
But you see that we don't know when they catch up.
The testing was completed in the study
at 8 1/2 years of age.
The question is, what happens when they get
to be 12, 15, 20?
The man whose picture I showed you earlier,
I think ramped up very quickly and was near
where normal-hearing age-matched peers are very quickly,
but there's a lot of variability.
Here are data from a study that we published last year,
in which we now looked at
also the IQ of the children.
So the children that come to my lab travel
from out of town, typically, with a parent.
The mothers typically have a very high education.
These kids come from a high socioeconomic background
with really intense habilitation
because to travel to Madison, Wisconsin,
from anywhere in the country,
for the parent to take time off work,
they really are gonna fall into that group.
So notice the mean IQ score
is down here in this black line.
Most of the kids in this sample fall above the mean in IQ.
So they're really smart kids, high ability to function
on these tasks.
But when we look at their language scores, they look
like the kids in the previous slide,
which is that they're very smart, but their language
is delayed.
So a lot of the kids are one standard deviation
or two standard deviations below the mean for their age.
So there's a lot of progress still to be made
even in this really high-functioning group of kids
who are getting sort of the best that they're going to get.
We also looked at something really fun to study,
which is real-time speech processing.
So, children are very good at processing language
in their native language.
And it takes time, though, if I'm going to show them
a picture of an object and then ask them to find a word
that they hear that matches that picture.
How long will it take them to visually identify
the auditory information and match the visual
and the auditory targets?
So I'm going to show you now a little video
of a 2 1/2-year-old boy who is wearing cochlear implants,
and he is looking at a screen that has the two pictures
that you see, a baby and a doggie.
We call it doggie.
And he's listening to the following sound.
So watch his eyes as the target word comes on.
I'll play it for you twice.
- [Voiceover] Look at the doggie?
Where is it?
- [Ruth] So you saw the video.
Let's watch it one more time,
and watch what his eyes are doing
as she's saying, "Where is the," which is not information,
and then after she says the target word, doggie,
what happens to his eyes.
- [Voiceover] Look at the doggie.
Where is it?
- [Ruth] So he finds the target.
His eyes land there.
And now this is what it sounds like with background noise.
(men's voices speaking simultaneously)
- [Voiceover] Look at the doggie.
Where is it? (cross-talk in background)
- [Ruth] So we call this +10 dB
signal-to-noise ratio,
which means that the target is 10 decibels
above that background babble noise.
Now we paid undergraduate students to analyze these data
by judging where the child was looking every frame,
30 frames per second,
and I'm gonna show you now the results.
This was done in collaboration with Jenny Saffran,
my colleague from the Waisman Center.
And now, what you see, is the proportion
of looking at the target.
So 50% would mean chance.
It means they're not looking at either the target
or the non target.
And then after the sound is presented,
how fast are they, how good are they?
Now, these are normal-hearing children.
The gray data are when the children--
Oops, sorry.
Are when the children are listening in quiet.
So notice that they get there fast,
and they get to nearly 100%.
They're not 100% because they're
not always paying attention when they're 2 1/2 years old.
These are the same normal-hearing children, now in noise.
So even if you're a normal-hearing 2 1/2-year-old,
the minute I introduce background babble,
something happens, and you're slower,
and you are less accurate.
Now, what happens to the children
who have cochlear implants?
I have grayed out the data from the normal-hearing children.
So the black triangles
are the cochlear-implant children listening in quiet.
They look like the normal-hearing children in noise.
And people say that listening to an implant
is like listening to a noisy system.
And that is one piece of evidence here,
which is that, even in quiet, at 2 1/2 years of age,
they've had at least one implant since 12 months of age,
they're still struggling.
And in noise, they're really having a hard time.
And those are the red triangles here.
So, the red diamonds.
So, yes, they can acquire language.
Yes, they can understand these.
The glass is half full.
But there's really room, also, for improvement
and for progress.
And these are the average data for the normal-hearing group
in quiet or with the competing sound,
the cochlear-implant group, which look like
the normal-hearing in noise, when they're listening
in quiet.
So we've had successes, and we've had limitations.
They perform with cochlear implants better
than children who have hearing aids
who have profound hearing loss,
but the variability is high,
and many of them are worse than their normal-hearing peers.
So the glass is both half full and half empty.
Well, what else might we need to worry about?
So, some of the hot terms today in the field
are executive function, which is how, what kind
of resources do you have in order to solve problems?
What is your working memory like?
What can you remember?
How fast can you solve problems?
What is your cognitive load?
How hard are you trying to solve problems?
These are very timely for people with hearing loss
because they come to the clinic often and report
that they're exhausted
from working so hard to listen all day.
A good example of somebody with two cochlear implants
is where the mother was telling me that
the girl used to go to school with one implant,
came home, was exhausted,
couldn't do after-school activities, slept.
When she got the second one, she said,
"Wow, there's a big difference,"
because she was working less hard.
And this is something that people with hearing loss,
with hearing aids will often report to their audiologist.
"I'm hearing the speech. I'm getting there.
"But, oh boy, I'm working so hard to get there."
So, we are trying to understand how speech understanding
and cognitive load are related.
And what we are doing to get there
is using a machine called an eye tracker.
What an eye tracker does,
if you see my research assistant here,
sitting in this contraption,
it's really not uncomfortable.
We're just holding her head still
because the cameras are closing in on her eyeballs,
and we are measuring where her eyes are looking
and the size of the pupils.
So, pupil dilation
has been, over the years,
suggested to be related to arousal, attention
to the environment, fight or flight.
It's controlled by norepinephrine and other things.
There's a big growing literature on this.
The idea is that, if your pupil is small, as is shown here,
you're exerting less effort.
But when you're exerting more effort,
the pupil size is larger.
So, we're bringing people into the eye tracking facility.
The procedure is, they listen to sentences
and they repeat them while they're fixating on this monitor.
And we measure real time processing, not unlike
what we did with the babies,
although the work that we did with the babies
was about eight years ago,
before we had the eye tracker.
Now remember earlier, I played some speech sounds for you,
the 4, 8, 16 channels?
This was the ...
(voice speaks distorted, metallic speech)
and then the natural.
- [Voiceover natural voice] Wanna come to my birthday party in August?
- [Ruth] Right, so now what we're looking at
are similar sentences that were degraded
to all of these levels but, as a function of real time
and a change in the pupil size in millimeters.
So we present the sentence.
After the sentence ends, the pupil size begins to grow.
And it reaches some kind of a maximum,
at the window in time when we think people
are rehearsing, processing, getting ready to tell us
what they heard, to give us the response.
And then it comes down.
The maximum pupil dilation is when the sound
is most degraded.
And the less degraded the sound is,
the less effort people are exerting.
So it's not just the percent correct.
It's not just that speech understanding changes.
It's that the pupil dilation, the listening effort,
the cognitive load is also changing,
and we can measure it in real time.
These are the summary of those data as a function of time
for the 4 channels, 8, all the way to the normal situation.
And again, these are the mean data for normal
and getting higher and higher and higher
as we get to the more degraded speech.
Okay, so the other area
that I mentioned several times
is really this question of whether children, adults,
should receive one or two cochlear implants.
And I did mention that fact, and you can see it here,
that, when we give a child two cochlear implants,
we're fitting them with two independent ears.
And it's really a wonder because they cost
about $60,000, so you'd think that by now,
we should have developed the engineering skills
to synchronize the two devices.
We can do that in my lab with a research processor
and in other people's labs, but actually, in the field,
it's harder than you think.
So one of the things we're working on
is to reverse engineer the way
that the brain uses information across the ears
so we can have these devices talk to one another
or be synchronized to improve performance.
But there is the question of one or both ears.
And remember, one of the things I mentioned earlier
was why binaural hearing, or being able to hear
with both ears is useful.
So, one of the usefulness areas
is sound localization.
Another one is the ability to separate speech from noise.
And the third one is what we call de-reverberation.
So we have localization, the cocktail party effect,
as we fondly like to call it,
and the idea that, if you're sitting in
a reverberant environment,
the binaural system can squelch or suppress echoes,
so that we can focus and give greater perceptual attention
or weight to the original sound.
And hearing in reverberative environments is something
that's very challenging for people with hearing loss,
and it's very challenging with only one ear.
So how do we study this?
I'll tell you just a little bit about some of these studies,
mostly by showing you these pictures
because the methods, I think, speak a thousand words,
or pictures speak a thousand words.
So if you have a little child who is about two years old,
we have to motivate them to participate on the task.
We've tried to do things like head turning
and eye gaze and really boring tasks,
and sometimes after you test them about 10 or 20 times,
they literally get up and say, "All done."
(laughs with audience)
Now, as the experimenter, you're not all done,
but they're done, and they've walked out the door. (laughs)
So we've designed a really fun task for them,
in which, what they're doing is, they're learning
that we hide a toy or sticker or food
behind a curtain,
and we present the sound from one of those locations,
and then they have to reach into the hole,
so it's called reaching for sound,
and they find the object of interest.
It's really like doing animal behavior studies. (laughs)
Where you're training the child,
you give them reinforcement,
and it works really beautifully.
And so they will sit there for an hour and a half,
and they do the task really beautifully.
But it took us a very long time to develop that
and to understand what 2 1/2 year-olds want
and how to test them.
When they get a little bit older,
we bring them into the sound booth,
and now this child is facing an array of loudspeaker,
and he is telling us, based on a computer game on a screen,
where he hears the sounds.
So he can have a spatial map, and he can tell us,
using this computerized game,
where the sound is coming from.
They can do this by about four or five years of age.
And then, when they're adults, we bring them
into other sort of more complex situations
where sounds can come from many, many locations.
They don't know where the sounds are,
and we use much more complex ways of testing them.
So, what we have found over the years is that bilateral,
or dual, cochlear implants significantly improve
the ability of children and adults
to know where sounds are coming from
and to understand speech and noise,
and we also have found that there is a reduction
in cognitive load.
And so I wanna, I have a few more minutes left,
so I wanted to show you how we measure the reduction
in cognitive load when we turn on either one ear
or the other ear or both ears.
Now most adults who have hearing loss will tell us
that they have a better ear
and a poorer ear.
They will say, "Oh my left ear,
"that's definitely much worse,"
Or, "Yeah, my right ear is much, much better."
So the question is, what happens if I turn on
just the poorer ear or just the better ear or both?
So here, what you see,
is the pupil dilation change
from baseline when we only turn on the poorer ear.
So you can see that, relative to stimulus offset,
they're listening to the stimulus,
the pupil is growing, growing, reaches a maximum,
they're rehearsing and repeating, and then it comes down.
When we only turn on the better ear,
it's a little bit better.
So there's a release from effort
when they're using their better ear.
They told us they like using that ear better,
and we can measure it with the pupil dilation.
Now, the question is, what happens when I turn on both ears.
Will the worse ear win by bringing it back up,
or will it be as good as just the better ear,
or will there be some kind of an improvement?
And we were delighted to find that, in fact,
there's some kind of an integration
that goes on in the brain, where having both ears,
even if one is worse than the other,
leads to a better ability to hear a speech signal.
So the cognitive load is reduced.
We knew from a lot of our studies
that they can get more words correct,
better percent correct.
But they're reporting to us
that they don't want to go home without both implants.
They don't want to leave the house
without both of their devices.
People are giving us those reports, subjective impression
of what life is like, and, in fact, it turns out
that we're measuring it with this reduction
in cognitive load.
Before I finish, I want to mention
that a number of years ago,
we had some very interesting things happen on campus,
and which I started to work on some of these issues
with my prior mentor, Tom Yin,
who conducted studies on sound localization
in cats for many years, and I know that many
of you followed the stories of what happened
with Tom's lab
and the People for Ethical Treatment of Animals,
and some of the controversy around this,
so what we tried to do is
to build a lab in which we could conduct some studies
in animals in which we provide them with cochlear implants,
understand what the brain is doing with those signals,
be able to then improve the signals,
so that we can then improve the engineering.
As many of you know, that didn't end up happening.
There were some political and controversial issues,
but I thought I would mention that Tom suggested
that I do that, and I'm happy to answer any questions
about how animal studies play
a role in a lot of what we have learned
about the bionic ear and the ability to provide sound
to people who are deaf through
this very complex technological advancement.
I wanted to finish off by saying
that bilateral listening mode produces
reduced listening effort.
Patient subjective reports of which ear is better
are consistent with the amount of listening effort.
And the bionic hearing, as we've followed it
from the 1970s and 1980s has really evolved
and has been improving.
People with profound hearing loss can hear and function
in everyday hearing environments.
We're working on improvement of the signal.
We think the importance of rehabilitation
and training really should continue to be mentioned.
We think it's important to reduce the cost.
And I really didn't discuss much here
about hearing culture versus deaf culture,
but I'd be happy to do that.
I wanted to thank everybody in my lab
and colleagues around the world and past students
who've been involved in this research.
Also, the work was funded primarily by
the National Institutes of Health, the National Institute
of Deafness and Communicative Disorders,
and also by the Waisman Center,
and various of the manufacturers who produce implants
have over the years helped us with some of the work.
Thank you for your time and attention.
(audience applauds)