There is a fixed value of 16px set for the Height of the GroupHeader. Thus, when a FontSize bigger than 16px is set, the Text is cut off as the GroupHeader is not resized. The workaround is to predefine the GroupHeaderContentTemplateSelector and remove the fixed Height that is set through it. 

<telerik:GroupHeaderTemplateSelector x:Key="GroupHeaderContentTemplateSelector">
                <telerik:GroupHeaderTemplateSelector.HorizontalTemplate>
                    <DataTemplate>
                        <ContentPresenter Content="{Binding FormattedName}"  Margin="4" />
                    </DataTemplate>
                </telerik:GroupHeaderTemplateSelector.HorizontalTemplate>

                <telerik:GroupHeaderTemplateSelector.VerticalTemplate>
                    <DataTemplate> 
                            <ContentPresenter Content="{Binding FormattedName}" Margin="4" VerticalAlignment="Top">
                                <ContentPresenter.LayoutTransform>
                                    <RotateTransform Angle="-90" />
                                </ContentPresenter.LayoutTransform>
                            </ContentPresenter> 
                    </DataTemplate>
                </telerik:GroupHeaderTemplateSelector.VerticalTemplate>
 </telerik:GroupHeaderTemplateSelector>