The following is an automatic (and unchecked) transcription of this talk:
So hi, I’m Sam. And this is the title is interpersonal entrainment during early social interaction. So we start with a paradox. Almost everything we know about how the brain functions during early social interaction comes from studying the brain in non interactive context. So, you know, I’ve done a lot of this research a lot, a lot, a lot of really, really brilliant, fantastic colleagues have done similar bits of research are going back to this paper from Teresa feronia, in 2001, where you flush up pictures of faces, and either looking directly at the camera or looking askance, and you measure infants brain responses to those images, another great experiment by a sushi send you and giggy CEBRA, where an actress starts looking down for exactly 4000 milliseconds, I think, looks up at the camera for 8000 milliseconds, then down at one of these two objects for 8000 milliseconds. And the same sequence repeats again and again. And this is another experiment by Yuka, Lebanon, looking at neural responses to different types of emotion expression. Yeah, so we found out a lot from these types of studies about how the infant brain processes social information. But all of these studies, you know, as having talked about this before, the author of these studies, you know, they readily acknowledge, you know, this is at a fundamental level, you know, not how the brain works during social interaction. Yeah, what we’re studying is a one way flow of information from the screen showing the picture, you know, often a static picture of a face to the baby. And social interaction, by definition, is bi directional. We also know that these bidirectional influences matter. So we know for example, that when a 16 month old initiates learning by pointing to an object, their memory for functions demonstrated in the object is increased. So that’s researching Katarina Begus, and Vicky Southgate.
We know that when a parent pays attention to an object, while they are engaged in shared activity with their baby, this immediately increases how attentive that baby is to that object. Yeah, that’s genuine Linda Smith, who found that. And we also know that the presence of an adult caregiver reduces the baby’s response to an experimental stressor, even if they’re not directly interacting with one another. And finally, we know that this affects learning too. So these by direction, inflammation factors affect learning too. So when nine month olds learn new speech sounds better to live in. So they learn speech sounds better to live interaction with an adult than watching a video. Yep. So that’s PAC, who showed up. So we know that social interaction is bi directional. And we know that this interaction makes a difference on how babies process information and how they pay attention and learn. Yep. But we have this problem at the moment, there are methods that were used to measure brain function in babies aren’t very well set up to, to measure these types of by direction influences. So what we’re doing is trying to understand why you’re trying to understand why we get these bi directional influences what the neural mechanisms observing them are. Okay. So first of all, we’re going to talk about why this type of work is relatively new, why it hasn’t been done before, because there’s a lot of challenges involved. And secondly, we’re gonna talk about two early findings that I’ve published or using it. And then we’re going to talk about what a really, really a lot of unanswered questions at the moment. Okay. So what we’re doing a lot of in my lab at the moment, is measuring do brain activity during free flowing naturalistic exchanges, in lots of lots of different contexts, as I get on to explain, but all of these naturalistic exchanges have a problem with it. So take this as one task, where we asked the mom to hold up a puppet on her hand, and to stage a conversation between the puppet and the child. And the child as they’re doing it is looking back and forth between the puppet and the child. So every time the child looks from the puppet to the for the parent, yeah, when we cut out their brain data, so the brain activity at the scope relative to this shift, yep. So this is what we’re looking at. So this is raw on process data. And we, we get this kind of patch of kind of a lot of activity around the side of the thing that we know is muscle artifacts. Yeah. So this is muscle artifacts associated with moving the head and moving the eyes at the same time. Yeah. And the method that we’re using EEG only picks up electrical activity, muscles work via electrical activity, just as neurons do an EEG current on its own tell the two apart. So we spent quite a lot of time work thinking at this looking at this problem. It’s a really specific problem for naturalistic studies because what we’re often interested in in naturalistic studies He’s not just, you know, a as in his screen paradigm where you flash up something and you look the pan out the brain activity relative to the flashing up. Yep. Because you know that if the baby’s moving or shoving around yeah, that we don’t expect a systematic relationship between the movement and occur the occurrence of a new image on screen, although some new there’s a new Val Greenberg neuron paper they would disagree with that. But generally, we can assume that the flashing up is independent of the movement. So your artifact is independent of the brain signal that you’re trying to look at. Whereas the naturalistic studies, often it’s the brain activity around a look that we’re trying to look at. Yeah. So this is a real problem. So we spent a long time trying to clean our data. So my PhD student I wrote merit hair science has done some really, really brilliant work on this, I published a paper in developmental cognitive neuroscience that looks really at the best kind of type of automatic cleaning that we can get. And even with manual cleaning, and with the very, very best ICA cleaning, we know that we can’t go into this artifact completely. This is really a problem with all EEG studies, by the way, anyone tells you that they’ve got rid of the artifact from their data in EEG, you know, whenever I’m reviewing a paper, I always say no, you haven’t. No, don’t be ridiculous. We know that we can’t clean data completely. But this is a it’s a real problem for us. So what we’ve got to do is we’ve got to try to be clever in how we analyze the data, we’ve got to think of ways that we can analyze the data that get around this problem. So this is 2030 seconds of a tabletop interaction, not like that, but like this, where we’ve got a series of different objects on the table. And we’re coding what the baby’s looking at, and what the moms looking at during this interaction. Yeah. So the blue are the segments where the baby is in attentive, the red at a time when they look to their partner. The yellow is when they looked at object one. And the green is where they look to object to and this is the mom doing the same thing. Yeah.
So first of all, what we can look at Yeah, is we can look at, for example, and all the times when the baby and the mom are paying attention to the same thing, and compare them to the times when the baby and the mom are paying attention to different things. And we can say separately, are they the differences in the amount of movement between the two conditions that might get in the way of that comparison. And then we can look at other types of things. So for example, here, this is a moment where the child looks to the mum, and then the mum joins the period of mutual gaze out. So by here, they’re both looking at each other. Yeah. And then this is a moment where the mum looks to the child at a time when the child isn’t looking to them. Yeah. So this is an add on gaze on sector mutual gaze. And this is an add on based on sector non mutual gaze. So we can compare the two. If we’re comparing it from the mums point of view, then we know that the eye artifacts around the movement should be the same. Yep. So if we get a difference between the two conditions after the look, then we can be sure as we can be that this isn’t to do with the eye artifacts, it’s to do with something else. Yeah. And then the third type of thing that we can look at is we can look at, so here there’s a moment where the child looks to object to and then the mum follows the child to object to. And here is a moment where the mum looks to object to and then the child follows the mum to object to Yeah, so this is an infant initiated look to object to and this is a mum initiated look. Okay? So again, we can compare the difference in brain activity between the two. If we time lock them both to the start of the infants look, then the infant neural activity should be the same, and any differences. So the infant’s ocular motor activity should be the same and any differences in neural activity, we can, we can be assured as we can be that they aren’t just to do with the fact that there is movement in there. Okay. So these are some of the methodological challenges associated with doing this work. And those are some of the solutions that we found to them so far. Now I’m going to talk about two early findings. So two, two, kind of early pieces of work. So this was work led by Vicki young, a longstanding collaborator of mine. And basically here, we recorded the infant and adult EEG, and at the same time, while the adult was either talking singing, directly reciting Nursery Rhymes directly to the infant, or looking 20 degrees to the side of them.
And then secondly, we did exactly the same thing. But instead of looking at live infant to live adult eg we did, we pre recorded an adult reciting some nursery rhymes to the camera, and we looked at the associations between the infants live EEG and the adults pre recorded EEG. So we just looked at C three and C four, which are two electrodes right over the vertex here, we calculated a generalized partial directed coherence, which is a cross spectrum measure of Granger causality. So you can look at how much one time series anticipates another time series in particular frequency ranges. So for this study, we just looked at a theater, which we were defining is three to six Hertz. And alpha, which we’re defining is six to nine hertz. So independently look just three to six hertz oscillator activity in the infant predict in the adult and vice versa. And then we look separately at adults. So this is complicated graph. But the takeaway is, during live interaction, we found that both adult to infant and infant to adult Granger causal influences were present in the three to nine Hertz range. Yeah. And we found that both influences was stronger during direct gaze compared to indirect case. Yeah. So that means that adults brain activity is anticipating the infant and the infant’s brain activities anticipating the adults, and that both relationships are stronger during direct compared to indirect case. Yeah. And we also looked separately, I haven’t got time to tell you about that. That is not because both a separately in training more to the speech that’s coming out of the adults mouth. Yeah, we suggested that it was independent of that. Yeah. And we also found during live interaction that the babies who vocalize for longer during direct gaze had a stronger Granger causal influence on an adult brain activity. So that’s suggesting some possibly suggesting some type of role for the infant vocalizations? And in driving these Granger causal inferences, okay. So that’s our finding of stronger sequential entrainment. So that is Granger causal, yet one brain activity predicting another brain activity in the three to nine Hertz range. And these are stronger during direct versus averted gaze. Okay? So when I present this to kind of audiences of neuroscientists, and they say, you know, that’s very well, and that’s very interesting. And it’s certainly a very striking finding. But Sam, how does it happen? Yeah, because we’re looking at oscillatory activity in the three to nine Hertz range, which is quite a fine grained timescale. How is it mechanistically? Possible? Yeah, for these two time series of brain activity to influence each other on such a fine grained timescale. And so that’s what we’re doing a lot of work on looking at at the moment. And we’re investigating a lot of different kinds of ways in which this might be true. And one of them is this idea that you might be getting phased resetting of neural oscillatory activity to gaze onsets, both in the person who looks at the child and at the time at the time of the person who’s being looked at. So that’s one hypothesis that we’re testing a variety of other ones to do to do with neural entrainment to the speech, and either other common features of the interaction. And And the other question that people ask when you present this as well, okay, that’s very nice. You know, we’re getting neural predictive activity. But so what why does this matter? Yeah. So in this paper, we’ve suggested one way in which it might be useful yet again, we don’t have any evidence for this yet. It’s just a hypothesis that we’re testing, which is that it could be that something around this synchronous activity means that when the adult, for example, is trying to teach something to the child, like they’re trying to present new word labels at a time for the child to memorize them, they find it easier to time those word labels at a time when the baby is maximally sensitive. Because we know that neural sensitivity varies with the phase of the oscillatory activity. And this type of Granger entrainment might make it easier for the parent to judge because they’re in training to the baby. When the moments of peak phase I what times are when the baby is most sensitive to these things that they’re trying to teach them. Okay? We haven’t shown that at all. That’s just a hypothesis. And we’re testing that at the moment.
So that was a little bit about the first kind of thing that we’re showing with these interactions. The second thing is, how does the brain activity relate to the behavior? Yeah, so this is the second new paper that I’m going to tell you about. Okay. So for this, we took a slightly different approach. So we have a child gaze and the adult gaze and the child EEG and an adult EEG. And first of all, we looked at the relationship between the baby’s attention and the baby’s brain activity. And to do that we did an analysis like this. So we took the brain activity at a particular frequency. So this is one to two hertz. And then we took the infant looking behavior. So this was just a simple one, they’re looking at objects or note they’re not. So in this time, they’re spending most of the time looking at the object with a few gaps in between. Yeah, we ran across correlation. And that’s basically looking at overall, and at times when the baby is looking at the object, is there brain activity in the one to two hertz range higher? Yep. And then the cross correlation part is if you shuffle one time series forward and backwards in time relative to the other, how quickly does this relationship drop off? Yeah, we did that for each baby independently, and then we did it for all babies, and then we average them. And then we drew that so I’m going to be building up this plot here. So that’s the first band Yeah, between one to two hertz on this range. Then we did the same thing for two to three hertz activity. Again, we calculated the cross correlate And again, we did it for all participants and averaged it. And then we added another line to our across to our final image. And then by the time we’ve done that, for all of the frequency ranges, up to about 16 hertz, we’ve got a plot that looks a little bit like that. And what this is showing us is theater activity. So activity in the five to eight Hertz range shows a strong attention with strong association with infant attention. Yeah. And then when we look this, so this relationship disappears when we when we break the synchrony looking five seconds forward in time and five seconds backwards in time. Okay. So that’s infant attention to infant brain activity. We also use the same approach to look at a parent attention to brain it to parent brain activity. Yeah. And then the really interesting thing that we think is when we look at the remote the social Chase association between parent attention and infant brain activity, so we look at the same relationship across the diet. Yeah. So this is I’ve shown you already this is infant EEG to infant gaze. Yeah, this is adult EEG to adult attention. So we get the similar pattern to what we find in the babies just at a high frequency range in the six to 14 Hertz range. But the really interesting plot, we think, is this one here, which is looking at the association between adult EEG and child case. Yep. And this is also showing a significant hotspot. Yeah, a significant pattern of association, where it looks like the babies, that the baby’s attention is associated with the adults gaze behavior. Yeah. So possibly, we think, during social interaction, adult neural activity in trains to infants attention, independent of their own attention. So I haven’t shown that but we did a another analysis in the paper showing that this enjoyment is independent of the adults attention is not just because the adults attention is tracking the infants attention. So that’s what I’m going to tell you for now. Infant adult brain to brain entrainment in the three to nine Hertz range is greater during direct versus averted gaze, that was the first paper. And then the second paper was during social interaction, adult neural activity and trains to infant behaviors independent of their own attention. Okay. So last part is just outstanding questions. So lots of these at the moment. This is very much early days for this type of research. The first is how does this happen, which I’ve already started to talk about, we’re doing a lot of work at that trying to understand the mechanisms why particularly relevant to the much more mature literature, neural entrainment to language? It’s a it’s a good question. Is this evoked activity? Or is this induced? Yep. So is this additive brain activity coming into my brain in response to things that you’re doing? Or is it induced? So is it changes in my brain activity? To match what’s happening in yours? Yeah. And a second question, and this is a paper from that looks at something similar? And is interpersonal entrainment affected by late and early stage processing? Or is it just specific to late stage or early stage? Yeah. Another question is, do we entrained to the interaction? Or does the interaction train to us? Yeah. So is the brain activity following the behavior? Or are we adapting our behaviors to fit in with the underlying patterns of brain activity? Again, this is a very interesting paper that looks at this question in a different way. And then we haven’t even started yet to think about an atypical development does it does a typical development manifest is atypical entrainment. Okay. Finally, another really important question, is this just an epiphenomenon? Or does it play a mechanistic role in intervention and learning? So that’s another thing that I mentioned earlier in the talk that we’re looking at in future. Okay. So just to say what I’ve said, and most current techniques for measuring physiology, physiological and brain activity during early attention, only look at a one way flow of information. Yeah, but this is a for poor fit for how social interactions work, we know that there are bidirectional mechanisms going on. So we need to work out what these mechanisms are at the neural level. naturalistic, unconstrained recordings have potential for telling us more than we know at the moment about how social influences operate dynamically, drawing attention and learning. Okay. We also talked about the technical challenges of analyzing naturalistic EEG and the ways around that that we’re finding at the moment. We talked about one paper suggesting that infant to adult entrainment is higher during direct versus aversive gaze, and another paper suggesting that through social interaction, adult neural activity in trays to infants attention independent of their own attention. Yeah. So these are the references in the papers that I’ve been talking about today. So if you’re interested, do feel free to look them up, or just drop me a line. But for now, thank you very much to funders, to colleagues and most importantly, to our research participants