Why is crowding released at the saccade goal? A commentary by van Koningsbruggen and Buonocore

mechanisms behind perisaccadic increase of perception

At the start of 2013, I published a paper in The Journal of Neuroscience showing that, just prior to a saccade, the deleterious effects of crowding are released at the saccade goal. “Crowding” refers to the phenomenon where an object in peripheral vision becomes difficult to recognise when it is closely surrounded by other objects. There’s a demo and a longer explanation of crowding and my previous paper here.

In this week’s issue of the same journal, Martijn van Koningsbruggen and Antimo Buonocore published a Journal Club paper examining the potential cause of a pre-saccadic release of crowding. The paper is behind a paywall here — email me for a copy if you don’t have access. It’s quite interesting to read how others interpret my work, and I think these sorts of commentaries represent and important stage of peer-review that occurs post-publication. Our formal response to this commentary has been published alongside the Journal Club paper, but here I’ll add some more thoughts about van Koningsbruggen and Buonocore’s explanation of our data.

In a nutshell, van Koningsbruggen and Buonocore’s main suggestion is that crowding may be released prior to a saccade because visual attention shifts to the saccade goal before the eyes move. The known relationship between eye movements and visual attention (e.g. Remington, 1980, Deubel et al, 1996) was part of my motivation for running the experiments in the first place. However, I don’t think pre-saccadic shifts of visual attention are adequate to explain our results, especially without a clear definition of “visual attention”. Our formal reply includes a summary of the specific reasons why “visual attention”, as it is used in the cognitive psychology literature, can’t fully account for our data. Depending on the authors’ definition of attention, there might be some explanatory power in their suggestion, but it seems to me that their use of “attention” actually describes an effect, such as a change in identification accuracy, rather than a mechanism, such as a change in the gain response settings of visual neurons.

To play devil’s advocate, I could, for example, argue that previous demonstrations of improved performance at the saccade goal also were the result of pre-saccadic changes in crowding, but have been called “attention” effects. Supporting this hypothetical argument, previous studies on eye movements and attention used stimulus configurations that would have been prone to crowding (e.g. Duebel et al, 1996, Kowler et al, 1995). My intention here is not to argue that this is in fact the case, but instead I’m trying to demonstrate that simply saying that changes in performance are due to “attention” may not necessarily encompass a meaningful explanation of the underlying neural mechanisms driving the changes in performance. Britt Anderson has written a great article about distinguishing “attention” as an effect versus cause in an open access article here.

Van Koningsbruggen and Buonocore bring up a few other interesting points about our study and its limitations which I mostly agree with, so it’s certainly worthwhile to read their article in full.

I’m very interested to carry on these discussions with other researchers, so please feel free to drop me a line or leave a comment here to share your thoughts. [Note that this was originally published on my old Harvard site, and there were a few contributions from people that may be worth reading: http://scholar.harvard.edu/willjharrison/news/why-crowding-released-saccade-goal-commentary-van-koningsbruggen-and-buonocore# ]

References

Our original article showing a release from crowding at the saccade goal:

Harrison, W. J., Mattingley, J. B., & Remington, R. W. (2013). Eye movement targets are released from visual crowding. Journal of Neuroscience, 33(7), 2927–2933. doi:10.1523/JNEUROSCI.4172-12.2013

Van Koningsbruggen and Buonocore’s response:

van Koningsbruggen, M. G., & Buonocore, A. (2013). Mechanisms behind Perisaccadic Increase of Perception. Journal of Neuroscience, 33(13), 11327–11328. doi:10.1523/​JNEUROSCI.1567-13.2013

Other references:

Anderson, B. (2011). There is no such thing as attention. Frontiers in Psychology, 2, 1–8. doi:10.3389/fpsyg.2011.00246

Deubel, H., & Schneider, W. X. (1996). Saccade target selection and object recognition: evidence for a common attentional mechanism. Vision Research, 36(12), 1827–1837.

Kowler, E., Anderson, E., Dosher, B., & Blaser, E. (1995). The role of attention in the programming of saccades. Vision Research, 35(13), 1897–1916.

Remington, R. W. (1980). Attention and saccadic eye movements. Journal of Experimental Psychology: Human Perception and Performance, 6(4), 726–744.

For a great introduction to crowding, as well as a heap of crowding demos, check out:

Pelli, D. G., & Tillman, K. A. (2008). The uncrowded window of object recognition. Nature Neuroscience, 11(10), 1129–1135.

Peer commentary: what does remapped crowding mean?


Along with James Retell, Roger Remington, and Jason Mattingley, I recently published a paper in Current Biology in which we describe “remapped crowding”. You can read about that article and effect here and here.

Denis Pelli and Patrick Cavanagh have just published a Dispatch in Current Biology discussing my article (link below), and what it means for trans-saccadic object recognition. I spent the better part of my PhD reading articles published by Pelli and (separately) by Cavanagh, so reading something they’ve written together about my work is novel.

Highlighted in the figure of their paper, Pelli and Cavanagh describe two ways in which remapped crowding may come about. Under both hypotheses, the location attributed to a target object is erroneously assigned to two different locations just prior to a saccade. One location is the object’s actual position, and the other location is the predicted, post-saccadic location of the object. The basic effect we described in our paper was that this object becomes difficult to identify when distractors are placed at the predicted location of the object. One possible explanation for such “remapped crowding” is that remapping may shift the representation of an object’s features prior to object recognition (Figure 1E of Pelli & Cavanagh). Target- and distractor-object elements appear jumbled and imperceptible because all features are necessarily mixed in a common (early?) processing area. Alternatively (Figure 1F), relatively accurately processed featural information may be drawn from two spatially separate locations simultaneously during remapping. Because the visual system is trying to make sense of multiple conflicting inputs, the ability to distinguish target and distractor becomes more difficult.

In our paper, in the second to last paragraph of the Discussion, we tended to favour the latter of the two suggestions because it seems more parsimonious based on previous work. However, the precise answer to the question will only come from further experimentation across a broad range of disciplines, and from independent lab groups. Some of my current work in psychophysics focusses on answering this question, but, for whichever hypothesis the behavioural research favours, we also need a plausible neural mechanism from the neurophysiology folk.

Here’s a link to Pelli and Cavanagh’s full article (email me if you don’t have access): http://www.sciencedirect.com/science/article/pii/S0960982213004302

Pelli, D. G., & Cavanagh, P. (2013). Object Recognition: Visual Crowding from a Distance. Current Biology, 23(11), R478–R479. doi:10.1016/j.cub.2013.04.022

(commentary on: Harrison, W. J., Retell, J. D., Remington, R. W., & Mattingley, J. B. (2013). Visual Crowding at a Distance during Predictive Remapping. Current Biology, 23(9), 793–798. doi:10.1016/j.cub.2013.03.050 )

Visual crowding at a distance during predictive remapping

My latest paper is now available online ahead of print via the Current Biology website. Click here to view the page. Please email me (willjharri AT gmail.com) to request a copy if you are unable to download via the site.

You can read my summary of the article here.

Harrison, W. J., Retell, J. D., Remington, R. W., and Mattingley, J. B. (2013). Visual crowding at a distance during predictive remapping. Current Biology 23, 1–6.
Available at: http://www.sciencedirect.com/science/article/pii/S0960982213003527.

Remapped crowding

I’m happy to write that I have just received notice that my most recent manuscript submission has been accepted and is now in press:

Harrison, W. J., Retell, J. D., Remington, R. W., and Mattingley, J. B. (2013). Visual crowding at a distance during predictive remapping. Current Biology.

In my last post, I had a little demonstration of “visual crowding”, the phenomenon where an object in peripheral vision becomes extremely difficult to identify when it is surrounded by visual clutter. You can also experience crowding by fixating the blue dot in part (a) of the figure below. Can you identify the Y buried amongst the Es? You might find that the parts of the letters appeared all jumbled. This is crowding. In part (b), fixate inside the blue dotted circle, and you’ll find it simple to identify the Y on the right side of the figure because it’s not crowded. In my previous paper, my co-authors and I showed that the deleterious effects of crowding (e.g. panel (a)) are reduced in the brief moments just prior to a “saccadic” eye movement toward the crowded object. That is, objects at the goal of the eye movement become easier to identify even before the eyes begin to move.

HarrisonRetellRemingtonMattingley-Figure01-v01

The main finding of this new Current Biology paper is that, under specific conditions, we get the opposite result: just prior to a saccade, an object in peripheral vision that is free from visual clutter, and therefore easy to identify, can be crowded by visual clutter quite distant from the object. This “crowding from a distance” occurs because of predictive remapping. Predictive remapping is a theory which suggests, just before an eye movement is made, the visual system predicts where visual objects will fall on the retina when the eye movement is complete. Similar predictions are made within our motor systems: when you plan to grab a cup of coffee on your desk, you plan to move your hand, and you have an expectation – a prediction – about the outcome of the plan (presumably to bring your hand to the cup). You can then compare the actual outcome of the movement with the prediction made before the movement to check if the movement achieved its goal. The same goes for movements of the eyes, but the outcome of the eye movement is new visual input.

In our study, we had people make a saccade to a specific location so that we knew where their eyes would move, and we thus had a good idea about the internal prediction regarding where things would fall on the retina following the eye movement. For example, in part (b) of the figure above, the observer would start by fixating within the blue dotted circle in the centre of the display. As shown by the orange arrow, they would then have to make a saccade (a fast eye movement) to the green dot at the far right. After they moved their eyes, observers had to report the identity of a letter presented briefly just before the eye movement began. In the case of the example, the letter Y would appear and disappear before the observer moved their eyes. The presentation of this letter probe was timed so that it appeared during the period of predictive remapping.

Because we knew the direction of the eye movement and the position of the probe, we also knew the predicted position of the probe. The predicted position of the probe letter is represented by the red arrow in the figure. That is, before an observer’s eyes moved, we knew where their visual system predicted the probe would appear on the retina following the eye movement. At this location, the probe’s “remapped location”, we presented visual clutter, the letter Es in this case. The positioning of the stimuli resulted in the probe becoming difficult to identify during, and only during, the period of predictive remapping, hence the title of the paper: “Visual crowding at a distance during the period of predictive remapping.”

The spatial arrangement of stimuli and timing of their presentation needs to be controlled tightly relative to a viewer’s eye movement, so it’s not feasible for me to create a demonstration of the effect. In the coming days (weeks?) I’ll update this post with a figure from the paper that schematises the logic and layout of stimuli. UPDATE: figure added.

More details and a link to the paper to come when it goes online.

If you’re interested in reading more about remapping, the following articles are good places to start:

Duhamel, J. R., Colby, C. L., and Goldberg, M. E. (1992). The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255, 90–92.

Merriam, E. P., Genovese, C., and Colby, C. L. (2003). Spatial updating in human parietal cortex. Neuron 39, 361–373.

Rolfs, M., Jonikaitis, D., Deubel, H., and Cavanagh, P. (2011). Predictive remapping of attention across eye movements. Nature Neuroscience 14, 252–256.

If you’re interested in reading more about visual crowding, try:

Bouma, H. (1970). Interaction effects in parafoveal letter recognition. Nature 226, 177–178.

Pelli, D. G., and Tillman, K. A. (2008). The uncrowded window of object recognition. Nature Neuroscience 11, 1129–1135.