- Welcome, all of you, to this symposium.
We've been planning this for well over a year
and it's really exciting to have this all come together
and to see you all here today.
And I'm really grateful to be able to be the first speaker
to kick off this set of talks that we'll be having
over the next few days.
So, today I'm gonna be talking about
communicating through color.
And the initial question I would like to pose is
why a symposium on color in the first place.
It turns out, color's a prime area
for interdisciplinary discourse
and a visual representation of what we just heard.
We bring together lots and lots of different departments
for this symposium.
So, we've got history, English, literary studies,
computer science, psychology, physics, and so on.
And so the question is why is it the case that color
is this prime area for interdisciplinary discourse?
Why does color bring us all together?
I'd like to suggest there's two main reasons,
at least from my perspective, why color brings us together.
The first is that colors activate a rich network
of concepts in people's minds,
and these concepts are highly flexible
and they can change, depending on the context.
So, for example, redness is associated with
the University of Wisconsin, but it can also signal
chemical processes in beets, it can tell us to stop
at an intersection, it can tell us about properties
of cells in imaging,
it can also tell us that a storm is coming.
So, concepts are activated by both perceptual experiences,
as well as verbal descriptions of color.
Given that it can be activated by perceptual experiences,
this brings together visually-oriented disciplines
and given that concepts can be activated
by verbal descriptions,
this brings together verbally oriented disciplines.
So, this brings together a lot of
differed disciplines together then.
Combining these points, color both perceived
and described is a catalyst for a thought, abstract thought,
across the arts, sciences, and humanities.
Okay, a second reason why I think color brings us together
is that color arises from sources that spans
the art sciences and humanities.
And I use the word "arises" very loosely because
I think we all would have very different definitions,
or many of us would have different definitions of what
color actually is across our different disciplines.
But I think we can probably mostly agree on the following,
which is that there's physical properties in the world
that predispose us to see colors.
There are biological systems that process
this physical input.
There are psychological processes in the mind
that integrate this input with prior knowledge.
And then there's things in the world,
there's things that create physical properties
and some of those things include humans,
which are probably gonna be influenced by these psychological processes.
So, when humans create art, design, imaging, and data visualization,
they're creating things in the world.
And there's also cephalopods,
which change color with camouflage.
Stars, plants, geological structures,
all of these are things that create physical properties
in the world that allow us to see color.
So, if you highlight each one of these areas,
we've got physics, biology, psychology,
art, design, imaging, data visualization,
cephalopods, stars, plants, and geological structures.
All of these things are involved in what we think of as color.
And so, I think that's another reason
why color brings us all together.
So, color, I think, helps us answer basic questions
in our respective fields,
but we also need these fields to answer basic questions
about color.
So, for the remainder of this talk,
I'm going to talk from the perspective of my field
in psychology about the questions I'm interested in
and how we go about answering these questions.
So, to begin, let's start with an exercise.
So take a moment to look at these crayons
and think about which color you like the best.
Okay, so now raise your hand if it was one of these colors.
Look around, see if there's a lot of you.
Okay, raise your hand if it was one of these.
Not many, and what about one of these.
Okay, a little bit fewer.
Most of them in the bluish range,
so we'll come back to that in a moment.
So, now we're gonna ask a different question.
Which college scholarship fund would you donate to?
Dennison College or Wilkes College.
Raise your hand if it's Dennison, so the one on the left.
Raise your hand if it's Wilkes, the one on the right.
This one's a little more distributed, I've never did this in public before,
that didn't work so well but that's okay,
I'll still make the point with data in a little bit.
We'll come back to that. (laughing)
Okay, one more, so which bin is for recycling glass?
So, here we've got a bunch of bins.
You've got a bunch of objects, take a moment to think about which object
would go in which bin, and I'm gonna ask you about a couple of them.
So, think about which one would be the right one for a glass.
Raise your hand if you think it's this one.
Really, someone from my, okay.
Raise your hand if you think it's this one. A lot more of you! (laughing)
So, it turns out that is what people generally think,
except for a person from my lab who knows
what these data are and he said the different one anyway.
So, these might seem like completely unrelated judgements.
The first one was about evaluating colors,
how much you like the colors.
The other one was about evaluating colleges,
which college you would donate to a scholarship fund.
And the third one was about throwing things away.
So, this is about interpreting color coding systems.
So, although these may seem like unrelated judgments,
I think there's an underlying commonality
and they all involve visual reasoning.
So, you use visual reasoning
to approach each one of these questions.
And I define this as making conceptual inferences
from visual information.
The visual information in these cases is color
and we'll talk more about what the kinds of conceptual inferences
you might have been making for these different kinds of tasks.
So, this brings us to the general question that we study
in my visual reasoning lab here,
at the University of Wisconsin in the Psychology Department
and the Wisconsin Institute for Discovery.
And our main question is how do people infer
meanings from colors and how do those inferences
influence people's judgements about the world?
So, to address this question,
we've been developing what we call
the Color Inference Framework to try to answer it.
The idea of the Color Inference Framework is that
this is a framework for how people form these
conceptual inferences from colors,
which we call color inferences.
And how those inferences influence judgements
about the world.
The framework is predicated on the notion
that people form color concept associations
from their experiences in the world.
So, what do I mean by this?
Well, let's actually start with a perceptual color space.
So, in a perceptual color space,
there are three dimensions, as most of you probably know.
Hue, which is the circular dimension,
like red, orange, and yellow.
Lightness, which goes from the bottom to the top,
so colors get lighter as they go higher in the space.
And chroma, which goes from the central spine outwards,
so colors get more pure and vivid
as they go further outward.
So, here's a perceptual color space,
and this space represents all the colors that we can perceive.
These are the samplings of the space but
internal of the space represents every color we can perceive.
Now, we can use this representation to talk about
color concept associations by building color
concept association spaces.
So, here we have a color space like before,
but now we have a weight on each point in this space
that represents how strongly that color
is associated with that particular concept.
So, here's a hypothetical color concept association
space for bananas.
So, there's a lot of weight on the yellows,
and the yellow greens, and browns.
There's very, very little to zero weight on reds and blues,
which is why you don't even see them in the diagram
because they're teeny tiny little orbs in there.
This is a color concept association space for ocean.
So, here there's a lot of weight around the blues
and blue greens, not much around the yellows,
which we don't generally associate oceans with yellows.
These are relatively peaky spaces,
meaning that a lot of the weight is concentrated
on one particular part of this space.
But other concepts would have much more
broadly distributed spaces.
So, for example, tropical birds.
Tropical birds can be associated with lots and
lots of different colors.
So, you're gonna have weight distributed across this space.
Now, the idea is that these spaces are continually
updated in the mind with new experiences in the world.
So, as we move about the world and experience new
color concept associations,
it can manipulate the weights in this space.
For example, a young child who only knows about
ripe bananas might only have weight on this
very saturated colors, but then as she learns about
rotten bananas and unripened bananas,
her color association space will broaden.
She also might make new spaces for ripe bananas
and rotten bananas that are different
from the overall banana space.
So, with these color concept association spaces that
we're constantly building and updating as
we move about the world,
we can construct a color concept association network.
And what this does is it's a network that represents
the association strength between all colors,
in this case, colors that are represented by squares,
and all concepts, which are represented by circles.
So, the idea is that all colors we can perceive
and all concepts we know about are linked together
through this color concept association network.
For example, a particular red might be associated
with the University of Wisconsin, roses, strawberry, and foot with a sore on it.
That looks really gross there, it's supposed to.
There are other things in this space that will
be really far away.
So, bananas will be really far away from red,
so it has very, very weak to zero weight
on the associations with the colors that are near red.
And our tropical ocean from before also
is very far away in this space.
But the idea is that this space connects all
these colors and concepts.
Now, going back to the framework.
The idea is that people can form conceptual inferences
from colors, or these color inferences,
which influence their judgements about the world.
And what I'd like to suggest is they do this by
using this color concept association network
and doing different operations on this same network
that leads to different kinds of inferences
and different kinds of judgements.
So, for example, a first type of operation
would be a pooling operation.
So, here the idea is that each one of the objects
that's associated with a color, we know things about them.
We have valences for them, so most of us probably
have a positive valence for the University of Wisconsin
and maybe for roses, unless you're allergic to them,
and probably a negative valence for the gross looking foot.
The idea then is that these valences get pooled onto
the color and we get a summary statistic of these valences
that is represented by that color.
When you go to make a judgment about the color,
it's informed by that summary.
So, if I ask you how much you like the color,
you're making an inference about all the objects,
the valences of the objects that are associated
with that color that get pulled onto the color
and that leads to your judgment about the color.
So, that's our first kind of operation.
Our second kind of operation is transmitting.
This is kind of similar to pulling but instead
of these valences just getting transferred onto the color,
they get transmitted to some other object
that is associated with that same color.
So, let's add a new object, this little greeble here,
you probably don't know much about this thing.
And the idea is that your valences for these objects
gets transmitted through the color as a conduit
onto this novel object.
And so, the judgment here is about the concept.
You judge how much you liked this greeble, so if I ask you
how much you like, based on how much you like
the things that are associated with that greeble.
And colors simply act as a conduit in this case.
We have valences from objects transferring to other objects
just based on their shared color association.
The third type of inference process is assigning.
And this one's a little bit different.
So, imagine you've got some colors and
you've got some concepts, and you're trying to figure out what the mapping is
between colors and concepts in a color coding system,
like that recycling example I gave you earlier.
The idea is that we figure out what the best assignments
are given the association strengths between the colors and the concepts.
And these lead to judgments about color concept mapping.
If I ask you which one of these bins is for paper and which one is for trash,
you'll be doing this assignment inference process
to figure out what that mapping is.
Okay, so to recap our operations,
we've got pooling, which leads to judgements about colors,
transmitting, which leads to judgements about concepts,
and assigning, which leads to judgements about
color concept mappings.
Starting with pooling, so the question here is
what determines people's color preferences?
Why do people have color preferences in the first place?
And why do people like some colors more than others?
And before I started working on this problem,
there were a few previous accounts.
One is that color preferences are hard-wired
in the visual system.
We have these weights on the cone visual system
that just somehow give rise to our color preferences.
But this account doesn't tell us why we have
the weights that we have or
why this would be hard wired in the way that it is.
Another previous account is that color preferences
can be predicted by color emotions.
And color emotions are these dimensions like
active-passive, cool-warm, heavy-light.
And it's been found that people like colors
that are more active, cool, and light.
But it's not clear why.
Why would people like colors that are more active
than passive, more cool than warm, and light than heavy.
It's also not clear that these are emotions.
So, these are the previous accounts of colors preferences
that were there prior to the work that we had done.
But back in 2010, Steve Palmer and I developed
the Ecological Valence Theory as a different account
for where color preferences come from.
And the idea is that you like colors that
remind you of things you like, like clear sky,
and ripe fruit, and rolling hills, and sunsets.
And you dislike colors that remind you things you dislike,
like the sewage water, rotting fruit, and that's mold,
and I spared you the picture of the vomit.
So, your preference for a given color then
is determined by how much you like all of the objects
that you associate with that color.
How can we test a theory like this?
In our initial study, we posed the question,
can we predict color preferences based on how much
people like the objects that are associated with those colors?
And we did this by testing four different groups of people.
So, the first group was presented with colors like these
and they were asked to write how much they like each color
on a scale from not at all to very much
by sliding a cursor and clicking to record their response.
A second group of people was presented with those same
colors but their task was to write down all of the objects
that they associated with those colors.
A third group was given the object descriptions that
were provided by the second group,
that was without colors, just the object description,
and their task was to write how positive versus negative
the objects were that were associated with each color.
And finally, a fourth group was presented with a color
along with the objects that were described
as associated with that color and
they rated how well the color on the screen
matched the colors of the objects.
And we did this to be able to down weight the effects
of poor matches and uprate the effects of good matches.
So, with these data, we were able to calculate what we
call a Weighted Affective Valence Estimate,
and what this is, is the single number that represents
how much people like all of the objects
that are associated with a particular color
weighted by how good the match is.
So, for each color, we have how much group one likes
those colors and then we have these weighted affected
valence estimates, which represent how positive
the objects are that are associated with the colors.
The question is, do these waves predict people's color preferences?
So, at first we're gonna look at color preferences.
So, here on the y-axis, we have mean preference ratings
and on the x-axis, we have hues,
so, red, orange, yellow, H is chartreuse, or yellowish green,
green, cyan, blue, and purple.
And here are what the data looked like,
and these different lines are for different saturation of lightness levels.
And there's two main patterns to take from this.
One is that there's this peak around blue.
Remember, most of you raised your hand for the bluish colors.
And in a trough around the yellowish green.
There's also a deep dip around dark yellow.
So, if you remember what things were associated
with dark yellow in the slide of pictures,
we had our rotting food and not picture of vomit.
So, now let's look at the waves.
Now, remember, the waves are measures of how much
people like objects associated with the colors.
The people who gave these data did not judge their color preferences at all.
And these are what the waves look like.
So, we have this peak around blue and the trough around yellowish green,
and the deep dip around dark yellow, along with other commonalities between them.
And these two data sets are highly correlated.
So, what this tells us is that we can indeed predict
how much people like colors based on how much people
like objects that are associated with those colors.
So, this is evidence for this pooling idea.
That your preference for color is influenced by how
much you like all these objects that are associated with the colors.
And I'm using the word influence but in this study, this is totally correlation.
So, we just find that there's this correlation
between color preferences and object preferences.
Can we provide evidence for a causal relation
where our color preferences actually caused by preferences
for objects that are associated with those colors?
And we have several studies that suggest that
that is the case but I'm gonna highlight one in particular.
So, what we did is we had people come into the lab
and we had them judge how much they liked each color,
just like the previous participants did,
but then we divided them into two different groups.
So, one group then saw positive red images,
like these bright raspberries and negative green images,
that's pond scum, and neutral other color images.
And the other group saw negative red images,
positive green images, and the same neutral other color images.
And when they saw these images, they saw them multiple times
and they did different kinds of tasks with them.
So, they were asked to click on the center to say
how complex they were, to judge how much they liked
the objects in the images,
and critically a different experimenter administered these tasks.
So, we presented to participants as there was this
color preference they needed first and then there's this
other spatial set of tasks that you're gonna do.
And then the original color preference experimenter came back in and said,
"Hey, can I get some more color preference judgements from you?"
And the participants would say sure,
and they did another round of color preference judgements.
So, this really did feel like two separate sets of experiments.
The participants generally did not know we were trying to manipulate
their color preferences and we debriefed them at the end
and if they had any idea that we were trying to, we excluded them from the analysis
but this was very few participants.
So, the question is can we actually change people's
color preferences by activating these associations
in their color concept association network
to influence their color preferences.
So, here we're looking at change in color preference
on the y-axis where positive means that people like
the color more after seeing the images.
So, people who saw positive red images and
negative green images had a significant increase
in their preference for red
and a minor decrease in their preference for green.
And people who saw positive green images
and negative red images had the opposite pattern.
The controller neutral colors were in between.
So, what this tells us is that color preferences
can be changed by experiences with positive
and negative-colored objects.
So, by priming objects in people's minds,
we can selectively manipulate preferences for particular colors.
We also, very recently, have neural evidence that
color object associations are involved in color preferences,
but I'm not gonna say any more about that because
you'll hear from Chris at 9:45 about this very new data.
Okay, so that was pooling
and so now we're gonna talk just a little bit about
transmitting and assigning.
So, for transmitting.
The question is does preference for a concept
that's associated with particular colors transmit
to concepts that are associated with similar colors?
We address this question in the 2014's Super Bowl.
So, during the 2014 Super Bowl,
the Seahawks, which were blue, green, gray and white
were playing the Broncos, which were blue, orange, red, and white.
And the question was does the team you're rooting for
during the Super Bowl-- and we collected these data online during the Super Bowl--
so people were watching and we know that they were watching because they told us they were.
Does your preference for these teams generalize,
or transmit to teams of division three colleges
where you don't know much of anything about them?
So, pilot data suggests that people don't know much
about Endicott College or Utica College.
And so the question was does your Super Bowl-preferred team,
your preference for that team,
spread to these division three colleges?
So, to test this idea, we had our two Division 3
colleges that were matched to the Super Bowl teams,
as well as other colleges that were matched to
other NFL teams, and we presented people with trials like this.
So, we said, imagine there are two minutes left in the game.
Who do you hope would win, in this case,
Endicott College or Utica College?
But they saw all pair-wise combinations of these teams.
And afterwards, we gave them a football questionnaire.
So, how much do you like the Seahawks and how much do you like the Broncos?
And the question is can we predict their choices
from this kind of bizarre task where they
don't know about these colleges
based on who they're rooting for in the Super Bowl.
We also did a similar thing in 2015
but instead of asking about which college team
they hoped would win in a Super Bowl, we asked which college
they would donate to their scholarship fund.
So, then we tried to move a little bit away from
the football context and we found similar effects there.
So, the question then is can we actually predict
which college they will choose, and we can.
So, here we're looking at NFL difference scores.
So, positive means that people like Seahawks more
and negative means they like Broncos more.
And the points are gonna be individual participant's choices.
So here, people who liked the Seahawks more were
more likely to choose the Endicott College
and people who liked the Broncos more were more
likely to choose Utica College.
Even though they don't know much about these colleges at all.
So, the idea is that your valence for a particular concept
is spreading to some other one through color as a conduit.
Now, this could have really serious implications
for the kind of judgements we make in the world.
For example, in college admissions.
If you're an admissions officer at Yale
and your rival is Harvard, are you gonna be more biased
to favor college applicants who have letterhead
from Exeter College, which is blue,
which matches your school's colors,
as opposed to St. Paul, which is red,
which matches the rival, Harvard's, colors?
So, there's a lot of cases where our associations
with colors might actually be influencing our judgements
in ways we don't realize because color is not relevant to the particular judgment.
We're activating this color concept association network
that's implicitly influencing our judgements.
So, this is a question we're following up with in the lab.
So finally, assigning, which gives us judgements about
color concept mappings.
So this is especially relevant when we're interpreting
color coding systems, and we do this all the time.
For example, when you're in an airport
and you've got different colored terminals,
you have to figure out which terminal maps onto which color.
When you're reading weather maps,
and when you're trying to figure out how to throw things
away in different kinds of recycling bins.
And it turns out that in different parts of the world
and even different parts of the same campus, you'll see different color coding systems
for different kinds of bins.
The question is what do these colors mean in these contexts?
More specifically, how do colors map onto concepts?
And you might be thinking, well this is a stupid problem.
Can't we just rely on legends or labels
to interpret what colors mean?
If there's some label that tells me what the color is,
then I can just read it and I know.
Well, it turns out that color coding systems are
harder to interpret when the labels
do not match people's predictions.
So, it's been shown by Lin, that if you show
people graphs of fictitious fruit sales,
they're much faster at interpreting the graphs
if bananas are yellow, and blueberries are blue.
Then if bananas are orange and blueberries are green,
that's even hard to say, because they are incongruent.
So, the question though is what are people's predictions
in these color coding systems?
Now, this might seem obvious when you're talking about
fruit because we know that bananas are generally yellow
and blueberries are generally blue.
But what about when there's abstract concepts
that do not have directly observable colors?
What are people's predictions there?
And what happens when you've got multiple concepts
that are associated with the exact same colors?
How do we figure out how to do the mapping
so that people can interpret the systems?
We've been working on what we call
Assignment Color Inference and the idea is that people
predict assignments that optimize
the color concept association.
So, they figure out in their minds,
they do an inference process where they figure out
what the best mapping is between colors and concepts,
given the colors that they have and the concept that they have.
And by understanding assignment inference,
we can actually create color coding systems that
are easier to interpret.
We'll hear a lot more about this later this afternoon
from when he talks about Modeling Color Inference:
A Study of Color-coding Systems in Recycling.
So, to conclude, I've suggested that a key reason
that many of us are united by color is that we use
color to infer and communicate abstract meaning
in our different disciplines.
And the color inference framework provides
an account for how.
It's an account for how people infer meaning
for colors and how those meanings influence people's
judgements about the world.
So, I'd like to acknowledge my collaborators on this work,
my lab and these wonderful people who have provided
valuable feedback on this work and funding sources.
And thank you for your attention.
(applauding)