To start this off, I will make a weak, straw man argument.
I propose a swiss army knife set of features of the environment are created by the brain, and then fed through an associative learner that dynamically weights them based on their magnitude (as a measure of confidence). Most of these features are 2 dimensional, which explains why painting look correct from multiple viewpoints. These features are line and shading based and allow us to be tricked by line drawings, stick figures or shading-only drawings.
For example, we have learned knowledge of the sizes and shapes of many different objects, in that we use this information for depth in a 2 dimensional image. We have an additional cue (focus depth) which can be used to make images or large things appear small to us -- the focus depth switches our expectations to a scene of miniatures rather than normally sized objects.
I don't know that I necessarily find that to be a weak straw man argument, Matt. Associative learning often seems like too small a hammer to attack large problems like vision, but while that may be true of computer vision, it might not be true of human vision.
Just think of how much visual stimuli we've all seen by the time we are 3-years old. Are we even close to training computer algorithms on this much data? Do we even have a practical way to collect this much data?
I think Cavanagh makes a good argument for the fact that vision might be simple yet elusive. I at least think we shouldn't gravitate towards complex theories over simple ones, until we have evidence that requires it.
I agree with what the Matts have said. Furthermore, I think it might help to look at the development of higher-level perception through the lens of studies of general human learning in Psychology. Looking at standard approaches, one of the most important (and experimentally measurable) methods for learning is association. As an example, association may help infants develop spatial perception by correlating feeling of edges of objects to seeing the lines and junctions that make up their visual image.
I also think that even advanced learning one performs as an adult can have a direct relationship to perception, leading to very real differences in the way individuals see the world. An oft-used analogy is the difference when looking at a chess board between novices and grandmasters- a novice will see a set of individual pieces, while the master will see much higher-level patterns that they have learned to recognize through years of practice.
I'm not sure whether I buy Cavanagh's argument that our ability to perceive simple unnatural stimuli means that we have simple internal representations that resemble those stimuli. I agree with Matt K, in that we shouldn't prefer complex theories unless there is sufficient evidence, but our ability to perceive complex stimuli points towards something at least a little more complex than what Cavanagh claims. The selectivity of neurons in higher-level visual areas is at a much finer resolution than outlines and sketches.
Furthermore, the task of reliably extracting a simple stick figure from an image is difficult (I think). So even though the final representation may be simple, the processing required to achieve this representation may be complex. In an associative network, you may end up with a deep hierarchical structure with complex intermediate features.
Although I can't cite many off hand, I feel like there are a lot of results from experimental psychology which point towards certain internal representations. For example, given the task of deciding whether two images represent different views of a 3d shape , the time to respond is a linear function of the rotation angle. This indicates that we are performing mental rotation of the 3d object, not just comparing the 2d sketches. I believe a working theory of vision should include internal representations which can support such manipulations.
I agree with the idea (that many have supported before me) that vision should be kept with a simple model over a complex theory. (without proof otherwise -- much like how we can quite easily overcomplicate things than need be -- the whole understanding a computer on Mars)
However, to many extents I like Cavanagh's relationship to how vision is simple due to art -- that we can infer many "blanks" due to previous held knowledge, and that we essentially have internal representations due to how we can "usually" correctly interpret art. I don't believe it would hurt for all components of vision to always consider art as being essential. (i.e. why can't we always detect a cartoony face if we train off real faces, but can identify a cartoony face -- do we have to have been told that this abstraction of a face is a face?)
I really like Matt K's idea that we get tons of "training" data from when we're younger. Just when and where do we learn and develop abstractions of what we know -- do we become more abstract in our associations of objects due to culture? Just a thought.
This line from Nakayama called my attention: "(Helmholtz) argued that our perceptions require unconscious inferences, linking the elementary sensations through locomotor experience to acquire the correct perception of size and distance". What is "correct" perception? Does it involve a classification scheme related to an ordinal measure between things we see in the world? Is it related to common perception (to what most of the people see) or to the "real" things that are out there?
This is a post about "Lightness, brightness and transparency...", which I am putting here because right now there doesn't seem to be a better place for it.
I found this reading very helpful in disambiguating terms such as lightness, brightness, luminance, illumination and reflectance. They all sound like they're talking about the same thing, but have meaningful distinctions. (As I currently understand it, lightness and brightness are percepts which account for the real properties illumination and reflectance. Luminance is a physical quantity we can actually measure)
I remain somewhat confused as to why transparency is often lumped in with lightness and brightness in the study of vision. While it's certainly true that transmittance and dispersion are important parts of light transport, it seems like we are biting off more than we can chew, so to speak. There are plenty of real world scenes where it would be perfectly reasonable to assume that all visible objects are opaque surfaces...
I found it very interesting to look at what questions vision researchers are asking--sometimes even moreso than what evidence they presented to support their answers. For example, take the question: "Does [perceiving] brightness precede, or follow layer decomposition?" There seem to be several assumptions here, some of which are probably well supported, and others of which might be worth challenging. We are assuming that... 1. the brain *explicitly* decomposes stimuli into LBT components 2. layer decomposition and perceiving brightness each happen once 3. layer decomposition and perceiving brightness come in a predictable sequence
I think assumption 2 may prove particularly troublesome. While it would be wonderful if it were true as far as understanding perception goes, it might not be a great evolutionary strategy. When it comes to the brain, redundancy and robustness trump efficiency.
I share Matthew's opinion about this article being useful for understanding better the differences between the terms. When I was reading I kept thinking about human learning processes as influential factors in perception. If I've seen an object many times under different illuminations, then it might be easier to differentiate between lightness and brightness for example. The effect of particular brightness illusions can even change with experience. I remember having seen some illusions that at first tricked me pretty easily, but that stopped having the same effect after a few more observations.
I feel like transparency is an important thing to consider when modeling our perception because it is like a border case, where traditional assumptions might fail.
Illusions seem to be an important topic to consider whenever you try to make a connection between how the brain performs visual processing. I like the idea the effects of illusions as you grow up -- if I recall correctly this would be in a way similar to how children have depth (and will at a certain age walk over the "edge" the glass cover of a table -- perhaps it would be interesting to see the effects of children seeing the various brightness illusions (not just depth) we commonly mistake -- (really young ages of course -- where we wonder if they "fill-in" brightness levels). However, I have to disagree with "being" used seeing an illusion so many times that they stop tricking me -- I still can seem to "turn on/turn off" seeing them. Maybe it's just me. Perhaps it is best to try and take completely "unexperienced" subjects and see the effects on children -- where they really couldn't have seen things before or at least for very long
Echoing Marynel, I really think that image statistics and prior experience play a large role in our perception of LBT. I like the example of Corney and Lotto's 2007 study which showed that training an ANN to idnetify reflectance on synthetic images led to errors similar to humans. Translating this to computer vision, I think it's important to train vision systems on large, natural image datasets if we want them to achieve human-like behavior. If these systems are trained on a set of images with different statistics, then the types of errors made may change and become inconsistent with human observers. In general, I believe incorporating priors on the natural world into computational models will continue to aid in our understanding of perception.
When reading the paper, I was not always convinced of how appropriate the translation was from the physical properties of luminance and reflectance to the percepts of lightness and brightness. In real scenes, we can easily distinguish the two if asked. However, in artificial illusions such as most of the demonstrations in the paper I saw little to indicate which was being tested for.
I do think that thinking in terms of perceptual analogs to the physical properties is a very good place to start- after all, we need something we can talk about that makes sense, and I certainly don't have better ideas at this point- but I can't help but think that there may be more nuanced theories for describing what we perceive.
In response to Peter, I think our definitions of physical properties are built partially out of our vision system in the first place, which is why they're a good place to start -- the unsensable properties came later in physics. It is unfortunate that glitching out the perception system to test edge cases of theories is our primary method of disproving or supporting them, as it was not the edge cases we were hoping to describe in the first place.
I was very surprised by the substantial differences in my perception of the different CCOB in figure 20 with and without the low frequency components. This, combined with the "default is sine wave" suggests multi-scale wavelet transformation of lightness information.
To start this off, I will make a weak, straw man argument.
ReplyDeleteI propose a swiss army knife set of features of the environment are created by the brain, and then fed through an associative learner that dynamically weights them based on their magnitude (as a measure of confidence). Most of these features are 2 dimensional, which explains why painting look correct from multiple viewpoints. These features are line and shading based and allow us to be tricked by line drawings, stick figures or shading-only drawings.
For example, we have learned knowledge of the sizes and shapes of many different objects, in that we use this information for depth in a 2 dimensional image. We have an additional cue (focus depth) which can be used to make images or large things appear small to us -- the focus depth switches our expectations to a scene of miniatures rather than normally sized objects.
I don't know that I necessarily find that to be a weak straw man argument, Matt. Associative learning often seems like too small a hammer to attack large problems like vision, but while that may be true of computer vision, it might not be true of human vision.
ReplyDeleteJust think of how much visual stimuli we've all seen by the time we are 3-years old. Are we even close to training computer algorithms on this much data? Do we even have a practical way to collect this much data?
I think Cavanagh makes a good argument for the fact that vision might be simple yet elusive. I at least think we shouldn't gravitate towards complex theories over simple ones, until we have evidence that requires it.
I agree with what the Matts have said. Furthermore, I think it might help to look at the development of higher-level perception through the lens of studies of general human learning in Psychology. Looking at standard approaches, one of the most important (and experimentally measurable) methods for learning is association. As an example, association may help infants develop spatial perception by correlating feeling of edges of objects to seeing the lines and junctions that make up their visual image.
ReplyDeleteI also think that even advanced learning one performs as an adult can have a direct relationship to perception, leading to very real differences in the way individuals see the world. An oft-used analogy is the difference when looking at a chess board between novices and grandmasters- a novice will see a set of individual pieces, while the master will see much higher-level patterns that they have learned to recognize through years of practice.
I'm not sure whether I buy Cavanagh's argument that our ability to perceive simple unnatural stimuli means that we have simple internal representations that resemble those stimuli. I agree with Matt K, in that we shouldn't prefer complex theories unless there is sufficient evidence, but our ability to perceive complex stimuli points towards something at least a little more complex than what Cavanagh claims. The selectivity of neurons in higher-level visual areas is at a much finer resolution than outlines and sketches.
ReplyDeleteFurthermore, the task of reliably extracting a simple stick figure from an image is difficult (I think). So even though the final representation may be simple, the processing required to achieve this representation may be complex. In an associative network, you may end up with a deep hierarchical structure with complex intermediate features.
Although I can't cite many off hand, I feel like there are a lot of results from experimental psychology which point towards certain internal representations. For example, given the task of deciding whether two images represent different views of a 3d shape , the time to respond is a linear function of the rotation angle. This indicates that we are performing mental rotation of the 3d object, not just comparing the 2d sketches. I believe a working theory of vision should include internal representations which can support such manipulations.
I agree with the idea (that many have supported before me) that vision should be kept with a simple model over a complex theory. (without proof otherwise -- much like how we can quite easily overcomplicate things than need be -- the whole understanding a computer on Mars)
ReplyDeleteHowever, to many extents I like Cavanagh's relationship to how vision is simple due to art -- that we can infer many "blanks" due to previous held knowledge, and that we essentially have internal representations due to how we can "usually" correctly interpret art. I don't believe it would hurt for all components of vision to always consider art as being essential. (i.e. why can't we always detect a cartoony face if we train off real faces, but can identify a cartoony face -- do we have to have been told that this abstraction of a face is a face?)
I really like Matt K's idea that we get tons of "training" data from when we're younger. Just when and where do we learn and develop abstractions of what we know -- do we become more abstract in our associations of objects due to culture? Just a thought.
This line from Nakayama called my attention: "(Helmholtz) argued that our perceptions require unconscious inferences, linking the elementary sensations through locomotor experience to acquire the correct perception of size and distance". What is "correct" perception? Does it involve a classification scheme related to an ordinal measure between things we see in the world? Is it related to common perception (to what most of the people see) or to the "real" things that are out there?
ReplyDeleteThis is a post about "Lightness, brightness and transparency...", which I am putting here because right now there doesn't seem to be a better place for it.
ReplyDeleteI found this reading very helpful in disambiguating terms such as lightness, brightness, luminance, illumination and reflectance. They all sound like they're talking about the same thing, but have meaningful distinctions. (As I currently understand it, lightness and brightness are percepts which account for the real properties illumination and reflectance. Luminance is a physical quantity we can actually measure)
I remain somewhat confused as to why transparency is often lumped in with lightness and brightness in the study of vision. While it's certainly true that transmittance and dispersion are important parts of light transport, it seems like we are biting off more than we can chew, so to speak. There are plenty of real world scenes where it would be perfectly reasonable to assume that all visible objects are opaque surfaces...
I found it very interesting to look at what questions vision researchers are asking--sometimes even moreso than what evidence they presented to support their answers. For example, take the question:
"Does [perceiving] brightness precede, or follow layer decomposition?"
There seem to be several assumptions here, some of which are probably well supported, and others of which might be worth challenging. We are assuming that...
1. the brain *explicitly* decomposes stimuli into LBT components
2. layer decomposition and perceiving brightness each happen once
3. layer decomposition and perceiving brightness come in a predictable sequence
I think assumption 2 may prove particularly troublesome. While it would be wonderful if it were true as far as understanding perception goes, it might not be a great evolutionary strategy. When it comes to the brain, redundancy and robustness trump efficiency.
I share Matthew's opinion about this article being useful for understanding better the differences between the terms. When I was reading I kept thinking about human learning processes as influential factors in perception. If I've seen an object many times under different illuminations, then it might be easier to differentiate between lightness and brightness for example. The effect of particular brightness illusions can even change with experience. I remember having seen some illusions that at first tricked me pretty easily, but that stopped having the same effect after a few more observations.
ReplyDeleteI feel like transparency is an important thing to consider when modeling our perception because it is like a border case, where traditional assumptions might fail.
Illusions seem to be an important topic to consider whenever you try to make a connection between how the brain performs visual processing. I like the idea the effects of illusions as you grow up -- if I recall correctly this would be in a way similar to how children have depth (and will at a certain age walk over the "edge" the glass cover of a table -- perhaps it would be interesting to see the effects of children seeing the various brightness illusions (not just depth) we commonly mistake -- (really young ages of course -- where we wonder if they "fill-in" brightness levels). However, I have to disagree with "being" used seeing an illusion so many times that they stop tricking me -- I still can seem to "turn on/turn off" seeing them. Maybe it's just me. Perhaps it is best to try and take completely "unexperienced" subjects and see the effects on children -- where they really couldn't have seen things before or at least for very long
ReplyDeleteEchoing Marynel, I really think that image statistics and prior experience play a large role in our perception of LBT. I like the example of Corney and Lotto's 2007 study which showed that training an ANN to idnetify reflectance on synthetic images led to errors similar to humans. Translating this to computer vision, I think it's important to train vision systems on large, natural image datasets if we want them to achieve human-like behavior. If these systems are trained on a set of images with different statistics, then the types of errors made may change and become inconsistent with human observers.
ReplyDeleteIn general, I believe incorporating priors on the natural world into computational models will continue to aid in our understanding of perception.
When reading the paper, I was not always convinced of how appropriate the translation was from the physical properties of luminance and reflectance to the percepts of lightness and brightness. In real scenes, we can easily distinguish the two if asked. However, in artificial illusions such as most of the demonstrations in the paper I saw little to indicate which was being tested for.
ReplyDeleteI do think that thinking in terms of perceptual analogs to the physical properties is a very good place to start- after all, we need something we can talk about that makes sense, and I certainly don't have better ideas at this point- but I can't help but think that there may be more nuanced theories for describing what we perceive.
In response to Peter, I think our definitions of physical properties are built partially out of our vision system in the first place, which is why they're a good place to start -- the unsensable properties came later in physics. It is unfortunate that glitching out the perception system to test edge cases of theories is our primary method of disproving or supporting them, as it was not the edge cases we were hoping to describe in the first place.
ReplyDeleteI was very surprised by the substantial differences in my perception of the different CCOB in figure 20 with and without the low frequency components. This, combined with the "default is sine wave" suggests multi-scale wavelet transformation of lightness information.