Does the same brain region help us recognize an inappropriate tool whether we hear about it or watch it being used?

Details

Although the neural correlates of how objects are used have been studied extensively, few studies have specifically examined the brain basis of knowledge about object function. This study aimed to assess supramodal semantic processes—meaning-related processes shared across sensory modalities—when people process actions involving familiar human-made objects presented visually or through speech.

The researchers hypothesized that common mechanisms support semantic processing of actions presented through different senses. Eighteen healthy male participants took part in an event-related functional magnetic resonance imaging (fMRI) study. The study used a 2 (modality) by 2 (congruence) factorial within-subject design, meaning each participant experienced the different combinations of presentation modality and whether the object suited the action.

In the Audio condition, participants heard short sentences describing actions with appropriate or inappropriate objects, such as “He writes with a pen/a button.” In the Video condition, they watched short clips of a person's arm performing common actions with appropriate or inappropriate objects, such as hammering a nail with a hammer or a fork. After each stimulus, participants judged whether the object-related action was congruent—that is, whether the object suited the action.

The researchers used a conjunction analysis to identify activation shared by spoken sentences and silent videos for incongruent actions relative to congruent actions in each modality. This analysis was intended to isolate semantic processing of actions themselves while controlling for lower-level perceptual processing and action-related motion or movement processing. To ensure that incongruent actions in both visual and auditory modalities contributed significantly, they inclusively masked the conjunction with the main effects of incongruent sentences versus rest and incongruent videos versus rest, each at a threshold of p=0.001. The statistical parametric map was thresholded at p<0.05, corrected for multiple comparisons.

The only area showing significant activation in this test was at the junction of left BA 44 and 45. The researchers interpreted this result as showing specific involvement of the ventral premotor cortex in modality-independent semantic processing of actions. They suggested that readily understood actions may be processed not only as a sequence in space and time but also in terms of meaning, possibly producing two distinct representations of the same action. The findings provide additional evidence consistent with a supramodal system involved in understanding actions.