This is reproducible also with the Green theme.
As a workaround you can extract the RadToolBar ControlTemplate and modify the column/row definitions of the root Grid element.
(http://docs.telerik.com/devtools/wpf/styling-and-appearance/styling-apperance-editing-control-templates)
This is for vertical orientation:
<Grid.ColumnDefinitions>
<ColumnDefinition x:Name="c0" Width="Auto"/>
<ColumnDefinition x:Name="c1" Width="*"/>
<ColumnDefinition x:Name="c2" Width="Auto"/>
</Grid.ColumnDefinitions>
<Grid.RowDefinitions>
<RowDefinition x:Name="r0" Height="Auto"/>
<RowDefinition x:Name="r1" Height="*"/>
<RowDefinition x:Name="r2" Height="Auto"/>
</Grid.RowDefinitions>
This is for horizontal orientation:
<Grid.ColumnDefinitions>
<ColumnDefinition x:Name="c0" Width="Auto"/>
<ColumnDefinition x:Name="c1" Width="*"/>
<ColumnDefinition x:Name="c2" Width="Auto"/>
</Grid.ColumnDefinitions>
<Grid.RowDefinitions>
<RowDefinition x:Name="r0" Height="*"/>
<RowDefinition x:Name="r1" Height="Auto"/>
<RowDefinition x:Name="r2" Height="Auto"/>
</Grid.RowDefinitions>