will j harrison

all about my active-vision research

me publications

Results for tag "predictive-remapping"

4 Articles

Peer commentary: what does remapped crowding mean?


Along with James Retell, Roger Remington, and Jason Mattingley, I recently published a paper in Current Biology in which we describe “remapped crowding”. You can read about that article and effect here and here.

Denis Pelli and Patrick Cavanagh have just published a Dispatch in Current Biology discussing my article (link below), and what it means for trans-saccadic object recognition. I spent the better part of my PhD reading articles published by Pelli and (separately) by Cavanagh, so reading something they’ve written together about my work is novel.

Highlighted in the figure of their paper, Pelli and Cavanagh describe two ways in which remapped crowding may come about. Under both hypotheses, the location attributed to a target object is erroneously assigned to two different locations just prior to a saccade. One location is the object’s actual position, and the other location is the predicted, post-saccadic location of the object. The basic effect we described in our paper was that this object becomes difficult to identify when distractors are placed at the predicted location of the object. One possible explanation for such “remapped crowding” is that remapping may shift the representation of an object’s features prior to object recognition (Figure 1E of Pelli & Cavanagh). Target- and distractor-object elements appear jumbled and imperceptible because all features are necessarily mixed in a common (early?) processing area. Alternatively (Figure 1F), relatively accurately processed featural information may be drawn from two spatially separate locations simultaneously during remapping. Because the visual system is trying to make sense of multiple conflicting inputs, the ability to distinguish target and distractor becomes more difficult.

In our paper, in the second to last paragraph of the Discussion, we tended to favour the latter of the two suggestions because it seems more parsimonious based on previous work. However, the precise answer to the question will only come from further experimentation across a broad range of disciplines, and from independent lab groups. Some of my current work in psychophysics focusses on answering this question, but, for whichever hypothesis the behavioural research favours, we also need a plausible neural mechanism from the neurophysiology folk.

Here’s a link to Pelli and Cavanagh’s full article (email me if you don’t have access): http://www.sciencedirect.com/science/article/pii/S0960982213004302

Pelli, D. G., & Cavanagh, P. (2013). Object Recognition: Visual Crowding from a Distance. Current Biology, 23(11), R478–R479. doi:10.1016/j.cub.2013.04.022

(commentary on: Harrison, W. J., Retell, J. D., Remington, R. W., & Mattingley, J. B. (2013). Visual Crowding at a Distance during Predictive Remapping. Current Biology, 23(9), 793–798. doi:10.1016/j.cub.2013.03.050 )

Visual crowding at a distance during predictive remapping

My latest paper is now available online ahead of print via the Current Biology website. Click here to view the page. Please email me (willjharri AT gmail.com) to request a copy if you are unable to download via the site.

You can read my summary of the article here.

Harrison, W. J., Retell, J. D., Remington, R. W., and Mattingley, J. B. (2013). Visual crowding at a distance during predictive remapping. Current Biology 23, 1–6.
Available at: http://www.sciencedirect.com/science/article/pii/S0960982213003527.

Remapped crowding

I’m happy to write that I have just received notice that my most recent manuscript submission has been accepted and is now in press:

Harrison, W. J., Retell, J. D., Remington, R. W., and Mattingley, J. B. (2013). Visual crowding at a distance during predictive remapping. Current Biology.

In my last post, I had a little demonstration of “visual crowding”, the phenomenon where an object in peripheral vision becomes extremely difficult to identify when it is surrounded by visual clutter. You can also experience crowding by fixating the blue dot in part (a) of the figure below. Can you identify the Y buried amongst the Es? You might find that the parts of the letters appeared all jumbled. This is crowding. In part (b), fixate inside the blue dotted circle, and you’ll find it simple to identify the Y on the right side of the figure because it’s not crowded. In my previous paper, my co-authors and I showed that the deleterious effects of crowding (e.g. panel (a)) are reduced in the brief moments just prior to a “saccadic” eye movement toward the crowded object. That is, objects at the goal of the eye movement become easier to identify even before the eyes begin to move.

HarrisonRetellRemingtonMattingley-Figure01-v01

The main finding of this new Current Biology paper is that, under specific conditions, we get the opposite result: just prior to a saccade, an object in peripheral vision that is free from visual clutter, and therefore easy to identify, can be crowded by visual clutter quite distant from the object. This “crowding from a distance” occurs because of predictive remapping. Predictive remapping is a theory which suggests, just before an eye movement is made, the visual system predicts where visual objects will fall on the retina when the eye movement is complete. Similar predictions are made within our motor systems: when you plan to grab a cup of coffee on your desk, you plan to move your hand, and you have an expectation – a prediction – about the outcome of the plan (presumably to bring your hand to the cup). You can then compare the actual outcome of the movement with the prediction made before the movement to check if the movement achieved its goal. The same goes for movements of the eyes, but the outcome of the eye movement is new visual input.

In our study, we had people make a saccade to a specific location so that we knew where their eyes would move, and we thus had a good idea about the internal prediction regarding where things would fall on the retina following the eye movement. For example, in part (b) of the figure above, the observer would start by fixating within the blue dotted circle in the centre of the display. As shown by the orange arrow, they would then have to make a saccade (a fast eye movement) to the green dot at the far right. After they moved their eyes, observers had to report the identity of a letter presented briefly just before the eye movement began. In the case of the example, the letter Y would appear and disappear before the observer moved their eyes. The presentation of this letter probe was timed so that it appeared during the period of predictive remapping.

Because we knew the direction of the eye movement and the position of the probe, we also knew the predicted position of the probe. The predicted position of the probe letter is represented by the red arrow in the figure. That is, before an observer’s eyes moved, we knew where their visual system predicted the probe would appear on the retina following the eye movement. At this location, the probe’s “remapped location”, we presented visual clutter, the letter Es in this case. The positioning of the stimuli resulted in the probe becoming difficult to identify during, and only during, the period of predictive remapping, hence the title of the paper: “Visual crowding at a distance during the period of predictive remapping.”

The spatial arrangement of stimuli and timing of their presentation needs to be controlled tightly relative to a viewer’s eye movement, so it’s not feasible for me to create a demonstration of the effect. In the coming days (weeks?) I’ll update this post with a figure from the paper that schematises the logic and layout of stimuli. UPDATE: figure added.

More details and a link to the paper to come when it goes online.

If you’re interested in reading more about remapping, the following articles are good places to start:

Duhamel, J. R., Colby, C. L., and Goldberg, M. E. (1992). The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255, 90–92.

Merriam, E. P., Genovese, C., and Colby, C. L. (2003). Spatial updating in human parietal cortex. Neuron 39, 361–373.

Rolfs, M., Jonikaitis, D., Deubel, H., and Cavanagh, P. (2011). Predictive remapping of attention across eye movements. Nature Neuroscience 14, 252–256.

If you’re interested in reading more about visual crowding, try:

Bouma, H. (1970). Interaction effects in parafoveal letter recognition. Nature 226, 177–178.

Pelli, D. G., and Tillman, K. A. (2008). The uncrowded window of object recognition. Nature Neuroscience 11, 1129–1135.

Eye movement targets are released from visual crowding.

Here’s a link to my recent Journal of Neuroscience article, coauthored by my PhD advisors Jason Mattingley and Roger Remington. We tested whether a target closely surrounded by distractors becomes easier to identify just before an eye movement is executed toward the target.

To explain the main findings in the paper briefly, consider the following. Look at the top cross on the left, and, without moving your eyes, see if you can identify the second letter in the string of letters at the far right. Now try identifying the letter in the second row while looking at the bottom cross.

crowdDemo

You’ll probably find it very difficult (impossible?) to identify the second letter in the top row, but it’ll be much easier to identify the same letter in the bottom row. The difference in difficulty can’t simply be due to the distance of each letter from the fixation cross, because the distance of each letter is the same. So, in the top row, there is some sort of interaction between the letters making it difficult to distinguish one from another. The difficulty recognising something in peripheral vision when it’s surrounded by clutter is referred to as “crowding”.

Back to my study. I had participants try to identify a crowded object. In half the trials, they kept their eyes still on a specific place, like the cross above, and they had great difficulty identifying the target as we expected. In the other half of the trials, the participants were required to make a saccadic eye movement toward the crowded target. Using an eye tracker, I was able to switch off the target prior to the start of each eye movement, so that the participants never saw the target after they moved their eyes. I found that, when participants had to identify the crowded target while also preparing an eye movement towards it, their ability to identify the target improved significantly. What’s most important about my findings is that this improved target recognition began before their eyes moved! So, preparing to make an eye movement resulted in better identification of a crowded target, effectively giving the participants a “sneak peak” of what’s in their peripheral vision even before their eyes moved. Pretty neat.

Harrison, W. J., Mattingley, J. B., & Remington, R. W. (2013). Eye movement targets are released from visual crowding. Journal of Neuroscience, 33(7), 2927–2933. doi:10.1523/JNEUROSCI.4172-12.2013