PET studies of the serotonin system in relation to behavioral phenotypes in psychiatry

Sammanfattning: Behavioral phenotypes beyond core symptoms of diagnoses have become an area of high interest in psychiatric research. All major psychiatric diagnoses are associated with deficits in cognitive function, including impairments in social abilities, which contribute to great suffering and have a negative impact on the everyday life. Aberrations in the central serotonin system has been demonstrated in several psychiatric disorders. However, treatment targeting cognitive impairment is lacking due to limited understanding of neurobiological mechanisms in psychiatry. The overall aim of this thesis is to investigate associations between cognitive ability and serotonin transporter and serotonin receptor 5- HT1B in major depressive disorder (MDD), autism spectrum disorder (ASD) and in nonpsychiatric control subjects. In Study I, ten patients with MDD were examined with cognitive tests and positron emission tomography (PET) and the radioligand [11C]AZ10419369 binding to 5-HT1B receptor before and after Internet-based cognitive behavioral therapy (ICBT) and for comparison with ten matched control subjects. The results showed improvements in verbal fluency from baseline to follow-up in the patient group. Correlations were found between improvement in verbal fluency and changes in 5-HT1B binding in ventral striatum and amygdala as well as between cognitive flexibility and dorsal brainstem, amygdala and hippocampus. In the control group when controlled for age and education level, an association between visuo-constructive memory and 5-HT1B availability in dorsal brainstem was demonstrated. The finding implicates a positive association between improvement in executive function and change in 5-HT1B binding in the MDD group. Study II investigated cognitive performance and the serotonin transporter (5-HTT) availability with PET and the radioligand [11C]MADAM in fifteen adults with ASD and fifteen matched control subjects. Analyses revealed lower 5-HTT availability in several brain regions in the ASD group compared to the controls. Also, positive associations between social cognition and 5-HTT binding were demonstrated. These results are in line with the hypothesis of lower brain 5-HTT binding in individuals with ASD, and further supports the theory of serotonin involvement in ASD neurodevelopment. In Study III, cognitive ability and 5-HT1B receptor binding with PET and the radioligand [11C]AZ10419369 were examined in 43 healthy control subjects. Our aim were to replicate parts of Study I as well as explore if other cognitive domains were associated with 5-HT1B. The findings between visuo-constructive performance and 5-HT1B receptor binding in the dorsal brainstem from Study I could not be replicated. Exploratory analyses when not controlled for age, revealed positive associations between visuo-constructive memory and 5-HT1B binding in several brain regions as well as negative correlations between 5-HT1B binding in numerous brain regions and cognitive flexibility and reaction time. When controlling for age effects, negative correlations between reaction time and 5-HT1B availability remained. Since a negative correlation between reaction time and amount of errors was found, implying faster reaction time and poorer impulse inhibition, these findings suggest that 5-HT1B receptors are involved in impulsive behavior. In Study IV, we investigated autism-related cognitive functions and the serotonin transporter (5-HTT) as well as serotonin receptor 5-HT1B with PET and the radioligands [11C]MADAM and [11C]AZ10419369 in a sample of healthy participants. In the 5-HTT sample, positive correlations between social cognition and standardized 5-HTT binding in striatum and putamen, as well as a negative correlation between social cognition and standardized 5-HTT binding in brainstem, were demonstrated. In the 5-HT1B sample, a significant correlation between central coherence and 5-HT1B binding in thalamus was found, but after controlling for age effects the correlation did not remain significant. The results of 5-HT1B binding in autism-like cognition do not support an association. Together with our finding of brain 5-HTT binding in relation to social-cognitive ability in neurotypical controls, and previous literature of individuals with ASD, a neurobiological and behavioral phenotype continuously distributed in the population is suggested for social ability.