Sensory Research: Neuroscience and Modelling

Journal Banner

Time course of face perception measured by differential Pavlovian conditioning

Yusuke Nakashima*, Yoichi Sugita

Department of Psychology, Waseda University, Tokyo, Japan

*Corresponding Author:
Yusuke Nakashima
Department of Psychology
Waseda University
Tokyo
Japan

Accepted date: September 06, 2017

Citation: Nakashima Y, Sugita Y. Time course of face perception measured by differential Pavlovian conditioning. Sens Syst J. 2017;1(1):1-9.

Visit for more related articles at Sensory Research: Neuroscience and Modelling

Abstract

We investigated the time course of face perception by using differential Pavlovian conditioning. Face and non-face images were used as conditioned stimuli (CSs). Some stimuli (CS+) were always followed by the unconditioned stimulus (UCS), whereas others (CS−) were presented alone. The UCS was an air puff delivered to the right eye. We measured the time necessary for discrimination between the faces by manipulating stimulus onset asynchrony (SOA) between the CS and the UCS. For discrimination between a face and a non-face object, successful differential conditioning was observed for short SOA. However, for discrimination between individual faces, differential conditioning was observed only when SOA was long. Moreover, much longer SOA was needed for discrimination between faces differing only in facial features or spacing. These results indicate that the time course of face processing stages can be observed by manipulating the SOA in differential Pavlovian conditioning. The time necessary for the discrimination measured by conditioning was shorter than that measured psychophysically, suggesting that conditioned responses are made through fewer processing steps than the subjective reporting in psychophysical measurement.

Keywords

Vision, Face perception, Pavlovian conditioning, Psychophysical measurement

Introduction

Face perception is an important skill in primates. Neuropsychological [1,2], neurophysiological [3], and functional neuroimaging [4-6] studies have demonstrated the existence of specialized brain areas for face processing. Face perception is accomplished through several distinct face processing stages. Face detection requires information about the basic spatial relationship between features of faces, for example, two eyes above a nose and the nose above a mouth [7]. Discriminating between individual faces requires more complex information such as subtle differences in the contour of faces, facial features, and spacing of the features [8].

In neurons of macaque temporal cortex, information about global categorization, i.e., human or monkey faces versus shapes, was conveyed in the earliest part of the responses. In contrast, information about fine categorization within each member of the global category, i.e., the identity or expression of faces, was represented later [9]. A magnetoencephalography (MEG) study also showed that a response occurring 100 ms after stimulus onset is related to face detection, while a response occurring 170 ms after stimulus onset is related to individual face identification [10]. These results suggest that face detection is processed earlier than individual face discrimination.

The present study used differential Pavlovian conditioning to measure the time course of face processing. Differential eyeblink conditioning was observed even in participants who showed no conscious awareness of the link between the conditioned stimulus (CS) and the unconditioned stimulus (UCS) [11,12]. Pavlovian conditioning is mediated by a low-level learning mechanism. Conditioned responses (CRs) are initiated without intention and may be unaffected by changes in observer’s criteria. By using differential eyeblink conditioning, tilt aftereffects were shown to occur in the spatiotopic reference frame, which are very weak and likely to be susceptible to decisional biases [13]. Measurement with Pavlovian conditioning would presumably observe pure sensory processing that occurs prior to decisionmaking. Typically, psychophysical measurements depend on the participant’s subjective judgments and cause subjective biases. Conditioning may be able to measure perceptual processing more objectively and sensitively than psychophysical measurements.

The time course of perceptual processing has been examined by manipulating the exposure duration of stimuli, using psychophysical methods [14,15]. In the technique, a stimulus is immediately followed by a mask to cut off processing of the stimulus. However, the masking cannot cut off all the processing stages [16]. The visual cortex is activated by the stimulus even after presentation of the mask [17-19]. The masking paradigm might not measure the time course of visual processing appropriately. We measured the time course of processing by manipulating the duration between the CS and the UCS in Pavlovian conditioning. This method would enable to observe only the processing from CS onset to UCS presentation.

The present study examined the time course of several processing stages for face perception, i.e., (1) discrimination between a face and a non-face object (object condition), (2) discrimination between faces of different persons (different condition), (3) discrimination between faces that differed in only the facial features (featural condition), and (4) discrimination between faces that differed in only the spacing of the features (spacing condition). The time course of these processing was measured by manipulating the duration between the CS and the UCS. In the tasks that needed higher-level processing, longer duration between the CS and the UCS were required for the acquisition of differential conditioning. We also measured the reaction times (RTs) using standard psychophysical measurement to compare performance between conditioning and psychophysical measurements. If Pavlovian conditioning is not involved in decision-making, participants would be able to discriminate the visual stimuli faster in conditioning measurement than in psychophysical measurement.

Experiment 1

To compare the time course between the psychophysical and conditioning experiments, the duration between the CS and the UCS in the conditioning experiment was determined based on RTs measured by the psychophysical method. The face discrimination tasks were conducted using psychophysical measurement in Experiment 1.

Methods

Participants

Twelve participants (21–27 years old, 6 females and 6 males) took part in Experiment 1. All participants had normal or corrected-to-normal vision, and were naive to the purpose of the experiment. The experiment was performed in accordance with the Declaration of Helsinki, and informed consent was obtained from all participants.

Stimuli

Gray-scale photo images of several female frontal view faces were prepared. From one of the faces, eight new versions were created. The four modified faces in the featural face set were created by replacing the eyes and mouth with the features of the same length from four different faces (Figure 1c). The two modified faces in the spacing face set were created by moving the eyes up and the mouth down, or the eyes down and the mouth up, relative to the original (Figure 1d). The distance between the eyes and the mouth was lengthened or shortened by 5%. Another two modified faces in the spacing face set were created by moving the eyes closer together or further apart by 5% relative to the original. Four different female faces were used for the different face set, in which the faces were different in features, spacing, and outer contour (Figure 1b). Another two faces were used for the object condition (Figure 1a). Grayscale images of two houses were also prepared for the object condition. The images (7° in width and 8.2° in height) were presented on a 24-inch CRT display (800 × 600 pixel resolution, refresh rate of 60 Hz) with a viewing distance of 65 cm. The mean luminance of each image was approximately 23.0 cd/m2.

sensory-systems-Different-face-stimulus

Figure 1: Design of Experiment 1. (a)-(d) Visual stimuli. (a) object stimulus set. (b) Different face stimulus set. (c) Featural face stimulus set. (d) Spacing face stimulus set. (e) Trial sequence. The fixation appeared for 800 ms, then the first stimulus was presented for 200 ms. After ISI of 200 ms, the second stimulus was presented and remained on the screen until participants respond. Participants judged whether the two stimuli were the same or different.

Procedure

A red fixation cross (16.3 cd/m2 and 0.8º in width and in height) was presented for 800 ms at the center of the screen (Figure 1e). The participants were instructed to look at the fixation point. The first face stimulus was presented for 200 ms. After 200 ms inter-stimulus interval (ISI), the second face was presented and remained on the screen until the participants responded. Both stimuli were presented at the center of the screen. The participants were required to judge whether the two stimuli were the same or different as soon and as accurately as possible by pressing the button. RTs were measured from the onset of the second stimulus to the participant’s response. In the featural, spacing, and different face conditions, the first and second stimuli were the same in half of the trials, and were different in the other half. In the object condition, the first and second stimuli were images in the same category in half of the trials, that is, both stimuli were faces or houses. In the other half, two images, which differed in category, were presented sequentially. In this condition, the participants were required to judge whether the categories of the two sequential stimuli were the same or not. The four conditions were tested in different sessions and in a random order between participants. Thirty-two trials were conducted in each condition. Four images in each condition were presented equally frequently and in a random order.

Results

RTs increased with complexity of the discrimination tasks (Figure 2a), with longer RTs for the discrimination tasks between faces than for the discrimination task between a face and a nonface object. Moreover, RTs for discrimination between faces of different persons were shorter than those for discrimination between faces in the featural set and discrimination between faces in the spacing set. However, RTs were not different between the discrimination tasks for the featural and the spacing faces. Repeated measures ANOVA revealed significant main effect of the stimulus factor (F3,33=16.30, p<0.001). RTs in the object condition were significantly shorter than those in the different (t11=3.54, p=0.02; p values adjusted by the Bonferroni- Holm method), featural (t11=5.97, p=0.0005), and spacing conditions (t11=6.26, p=0.0004). RTs in the different condition were shorter than in the featural (t11=3.29, p=0.02) and spacing conditions (t11=3.23, p=0.02). RTs in the featural and spacing conditions were not different (t11=0.26, p=0.80).

Accuracy was measured as the proportion of the correct responses for both trials in which the two sequential stimuli were the same and different (Figure 2b). Accuracy for the discrimination task between a face and a non-face object was almost 100%. Accuracy in the three tasks for discrimination between individual faces was around 80%. Repeated measures ANOVA revealed significant main effect of the stimulus factor (F3,33=13.63, p<0.001). Accuracy in the different, featural, and spacing conditions was not different (different vs. featural: t11=0.24, p=1.0; different vs. spacing: t11=0.88, p=1.0; featural vs. spacing: t11=0.61, p=1.0). Accuracy in the object condition was higher than that in the different (t11=5.86, p=0.0006), featural (t11=5.91, p=0.0006), and spacing (t11=4.08, p=0.007) conditions. The p values were adjusted by the Bonferroni-Holm method for the comparison between the stimulus conditions.

sensory-systems-stimulus-conditions

Figure 2: Results of Experiment 1. Average data across the participants are shown. Error bars indicate the standard error. (a) RTs (ms) in the four stimulus conditions. (b) Accuracy (%) in the four stimulus conditions.

Experiment 2

In Experiment 2, we measured discrimination performance by using differential eyeblink conditioning. To examine the time course of face processing, we manipulated stimulus onset asynchrony (SOA) between the CS and the UCS. To compare the time course between the psychophysical and conditioning experiments, we used SOAs that are matched to the RTs for the discrimination tasks in Experiment 1.

Methods

Participants and stimuli

The same participants from Experiment 1 took part in Experiment 2. Furthermore, the same images of face and nonface objects as in Experiment 1 were used.

Procedure

A red fixation cross was presented for 1,000-1,500 ms (randomly varied) at the center of the screen, followed by the presentation of a face or house stimulus. One or two of the four stimuli in each stimulus set (CS+) were always followed by the UCS, whereas the other stimuli (CS-) in the set were presented alone. The UCS consisted of a 100-ms, 5-psi air puff, and was delivered to the right eye via flexible plastic tubing terminating in a 1-mm nozzle placed 45° right of the right eye. The distance between the nozzle tip and the cornea was approximately 3 cm. The SOA between the CS+ and the UCS was 500, 600, 700, and 700 ms in the object, different, featural, and spacing conditions, respectively (Figure 3). We used approximately the same duration as RTs for the face discrimination task in Experiment 1. We also tested two other types of SOAs: one was 200 ms shorter than the RT-matched SOA (300, 400, 500, and 500 ms) and the other was 300 ms shorter than the RT-matched SOA (200, 300, 400, and 400 ms).

sensory-systems-temporal-relationship

Figure 3: The temporal relationship between the CS and the UCS. In the RT-matched SOA condition, SOAs between the CS and the UCS were 500, 600, and 700 ms in the object, different, and featural and spacing conditions. In the 200 ms-shorter SOA condition, SOAs were 200 ms shorter than those in the RT-matched SOAs. In the 300 ms-shorter SOA condition, SOAs were 300 ms shorter than those in the RT-matched SOAs. The UCS was presented for 100 ms. The CS and the UCS coterminated.

The CS and the UCS terminated simultaneously. The inter-trial interval (ITI) varied randomly in the range of 4 to 7 s. The color of the fixation cross was often red (x=0.63 and y=0.35 in the CIE chromaticity coordinates), but a fixation of bluish red color (17.7 cd/m2; x=0.36 and y=0.20) was presented with a 25% probability. Participants were instructed to press a button when the bluish red fixation appeared in order to make sure that they fixate the center of the screen.

Eye blinks were recorded using pairs of electromyographic (EMG) electrodes (4 mm Ag-AgCl). The electrodes were placed on the orbicularis palpebrarum muscle below each eye with the ground and reference electrodes on the earlobes. EMG data were recorded at 1 kHz. An eyeblink occurring from 200 ms before UCS onset to UCS onset, the amplitude of which was equal to or greater than 10% of the average amplitude in response to the UCS presentation in the same session, was counted as an eyeblink conditioned response (CR). To monitor the eye movements, electrooculography (EOG) was also measured with two electrodes placed near the outer canthi of both eyes. Trials in which participants made eye movements while the stimuli appeared were rejected.

Participants underwent another 12 sessions (four stimulus conditions × three SOA conditions). The sessions were tested in a random order between participants. Each session consisted of four blocks and each block included six CS+ and six CS− trials. In all three conditions (different, featural and spacing), an image of the CS+ was presented six times in the six CS+ trials, while the other three images were presented two times in the six CS− trials. Trials were presented randomly, with no more than two trials of the same type in a row. A different image was assigned to the CS+ between participants and was counterbalanced. In the object condition, the two face images were used as the CS+ and the two house images were used as the CS− in half of the participants. In the remaining half, the CS+ and the CS− were reversed. The two types of face images or house images were presented randomly in each block.

Results

Differential CRs were measured as the percentage of CRs to the CS+ minus the percentage of CRs to the CS− (Figure 4). Differential CRs did not increase as the training progressed for all the stimulus conditions in the 300 ms-shorter SOA condition (Figure 4a). However, differential CRs increased for all the stimulus conditions in the 200 ms-shorter SOA condition as well as in the RT-matched SOA condition (Figures 4b and 4c).

sensory-systems-Error-bars-indicate

Figure 4: Results of Experiment 2. Average data across the participants. Error bars indicate the standard error. (a-c) Differential CRs (%) for each block in the four stimulus conditions. (a) The 300 ms-shorter SOA condition. (b) The 200 ms-shorter SOA condition. (c) The RT-matched SOA condition. (d) Differential CRs (%) for the later part of the training (the average of the 3rd and 4th blocks) as a function of SOA. Data for each stimulus condition are shown. A double asterisk indicates significance at p<0 .01 (compared with zero).

We plotted differential CRs for the latter part of the training (the average of the third and fourth blocks) as a function of SOA (Figure 4d). Participants exhibited successful differential conditioning in all the stimulus conditions even when the SOAs were 200 ms shorter than RTs for the discrimination task in Experiment 1. However, when the SOAs were furthermore 100 ms shorter, differential CRs dropped to approximately zero. Differential CRs were significantly larger than zero in the object-300 ms (t11=4.55, p=0.002; p values adjusted by the Bonferroni-Holm method), different-400 ms (t11=5.50, p=0.0006), featural-500 ms (t11=3.73, p=0.003), and spacing-500 ms (t11=6.38, p=0.0002) conditions. However, they were not larger than zero in the object-200 ms (t11=0.69, p=1.0; p values adjusted by the Bonferroni-Holm method), different-300 ms (t11=0.77, p=1.0), featural-400 ms (t11=0.48, p=1.0), and spacing-500 ms (t11=0, p=1.0) conditions. I n the RT-matched SOA condition, differential CRs in all the stimulus conditions were larger than zero (object-500 ms: t11=5.50, p=0.0006; different-600 ms: t11=7.52, p=5E-5; featural-700 ms: t11=4.95, p=0.0009; spacing-700 ms: t11=4.36, p=0.001; p values adjusted by the Bonferroni-Holm method)

When the SOA was 400 ms, differential CRs in the featural and spacing conditions were zero, but those in the different condition were still large. Similarly, when the SOA was 300 ms, differential CRs in the different face condition were zero, but those in the object condition were still large. Differential CRs in the different-400 ms condition were larger than those in the featural-400 ms (t11=5.01, p=0.0008; p values adjusted by the Bonferroni-Holm method) and spacing-400 ms (t11=4.42, p=0.001) conditions. Differential CRs in the object-300 condition were larger than those in the different-300 condition (t11=4.57, p =0.0008). These results indicate that conditioning can measure the difference in task complexity by varying the SOA between the CS and the UCS.

Experiment 3

In Experiment 3, the discrimination tasks were conducted by using the psychophysical measurement again. To examine whether participants can perform the discrimination tasks within the duration in which they could acquire differential conditioning when using the conditioning measurement in Experiment 2, we instructed participants to judge with time constraints.

Methods

Participants and stimuli

The same participants as in Experiment 1 and Experiment 2 took part in Experiment 3. Moreover, the same stimuli were used as Experiment 1.

Procedure

Experiment 3 was the same as Experiment 1, except that participants had to respond at a given time. The first stimulus was presented for 200 ms. After 200 ms ISI, the second stimulus was presented for a specific duration, then disappeared. The exposure duration of the second stimulus was the same as the 200 ms-shorter SOA in Experiment 2: 300, 400, 500, and 500 ms in the object, different, featural and spacing conditions, respectively. Participants were required to judge and press the button just when the second stimulus disappeared. Before each session, participants were trained to respond at the correct timing.

Result

Participants could respond at the correct timing (Figure 5a). RTs in the object, different, featural, and spacing conditions were not significantly different from 300 (t11=0.43, p=0.67), 400 (t11=0.87, p=0.40), 500 (t11=0.46, p=0.66), and 500 (t11=0.84, p=0.42) ms. Accuracy decreased to near the chance-level (50%) for all the stimulus conditions when the time constraint was provided (Figure 5b). Accuracy was not significantly different from 50% in the object (t11=1.80, p=0.15; p values adjusted by the Bonferroni-Holm method), different (t11=1.96, p=0.15), featural (t11=2.43, p=0.10), and spacing (t11=2.61, p=0.10).

sensory-systems-standard-error

Figure 5: Results of Experiment 3. Average data across the participants. Error bars indicate the standard error. (a) RTs(ms) in the four stimulus conditions. (b) Accuracy (%) in the four stimulus conditions.

Discussion

The present study examined the time course of face processing by using differential Pavlovian conditioning. Differential conditioning was successfully acquired for the discrimination between a face and a non-face object even when SOA between the CS and the UCS was short. However, for the discrimination between individual faces, which requires higher processing, differential conditioning could be acquired only at longer SOA. These results indicate that Pavlovian conditioning can measure the time course of visual processing by manipulating SOA between the CS and the UCS. Furthermore, the time necessary for the discrimination measured by conditioning was shorter than that measured psychophysically.

The hierarchy and the functional stream of visual processing have been greatly revealed by measuring neural activities. Cortical location and time course of neural correlates are critical evidence to elucidate detailed mechanism of the visual hierarchy. However, the visual input is processed not only by feed-forward projections, but also by feedback projections [16,20]. Sensory signals are unlikely to flow sequentially through the serial pathway, but rather flow in a complex recurrent network. As a consequence, there arises difficulty in estimating the hierarchy of visual functions only by examining the cortical location of neural correlates. Measuring the latency of neural activities is also insufficient because the observed activities might not reflect completion of processing but an intermediate stage. Moreover, the visual response latency of the cortical neurons is broadly distributed and largely overlaps between different visual areas [21]. Recurrent processing has been reported to be involved in several functions of visual perception [22-28]. Face recognition is also involved in feedback projections. Several face patches in the temporal cortex of monkeys are anatomically connected to each other and higher areas provide feedback projections to lower areas [29]. These previous findings suggest that behavioral responses also need to be examined for understanding the hierarchical mechanism of the visual pathway.

In psychophysics, the time course of visual processing is investigated by manipulating the exposure duration of stimuli, in which a stimulus is followed by a mask to cut off processing of the stimulus [14,15]. This paradigm assumes that the masked stimulus is not processed in any subsequent stages. However, masks do not catch up with initial activities of feed-forward responses. Short responses are evoked by the stimulus in the lowlevel [17] and high-level [18,19] visual areas after presentation of the mask. Therefore, a method that manipulates the exposure duration by masks is not sufficient to measure the time course of processing appropriately. The present study measured the time course of face processing by manipulating SOA between the CS and the UCS in Pavlovian conditioning. In our method, visual processing at the time when the UCS is presented might be observed.

The hierarchical order of face detection and individual face discrimination are demonstrated by the time course of neural activities. Studies that measured single unit activity in the temporal visual cortex of monkeys [9] and MEG responses in the occipito-temporal cortex [10] have shown that face categorization or detection is represented in the earliest part of the responses, whereas facial identity is represented in the latter part. The present study showed that successful differential conditioning for discrimination between a face and a non-face object was achieved when SOA between the CS and the UCS was longer than 300 ms. In contrast, for the discrimination between individual faces, differential conditioning was successfully achieved when SOA was longer than 400 ms. These results are consistent with the findings that face detection is mediated at an earlier processing stage than individual face discrimination.

Differential conditioning for different individual faces was acquired with shorter SOA, while longer SOA was necessary for conditioning for faces that differed in only features or spacing. This is consistent with previous results reporting the faces that differed in both external and internal features were discriminated more easily than the faces that differed in either external or internal features [30]. However, the SOA necessary for successful differential conditioning for faces differing in features was not different from that required for faces differing in spacing.

The time course measured by conditioning was approximately 200 ms faster than that measured psychophysically, suggesting that conditioned eyeblink responses are made through fewer processing steps than the psychophysical reporting such as button press. It has been shown that Pavlovian conditioning occurs independently of consciousness. Differential eyeblink conditioning [11,13] and fear conditioning [31] were acquired without conscious awareness of the relationships between the CS and the UCS. Fear conditioning can be acquired even when the CS is completely suppressed from awareness [32]. Eyeblink conditioning is mediated by a low-level learning mechanism, for which the loci responsible include the cerebellum and the brainstem [33]. It has been confirmed that visual CS pathway from the visual cortex to the cerebellum for eyeblink conditioning exits [34]. A recent study employing Pavlovian conditioning to measure the strength of aftereffects reported that tilt aftereffects occur in spatiotopic reference frames, which are susceptible to decisional biases [12]. Taken together, these results suggest that Pavlovian conditioning may occur without the stage of decision-making. Subjective judgments used in psychophysical measurements cause response and decisional biases. By using measurement with conditioning, we might be able to examine sensory processing more directly and objectively than by psychophysical measurement.

Acquisition rates in eyeblink conditioning as a function of the time interval between the CS and UCS have been studied using simple sensory stimuli such as tone and light [35-38]. Largest percentage of CRs seems to be obtained when the time interval is approximately 400-500 ms, and CRs decrease when the interval is shorter or longer than the appropriate time. In the present results, the differential-CRs curves as a function of SOA shifted depending on the stimulus conditions. This suggests that appropriate time interval between the CS and UCS varies depending on complexity of the CS and that the appropriate time reflects the time necessary for perceptual processing of the CS.

Acknowledgements

This work was supported by JSPS KAKENHI Grant Numbers JP16H07284.

References