Basing on the "{StaticResource FilterControlStyle}" would cause an InvalidArgumentException, if the types would mismatch. If you are to base on a custom style on the default style of either of those controls, you can do it by type reference - e.g. BasedOn="{StaticResource {x:Type telerik:FilteringControl}}"
The fix will be available in the R1 2018 SP2 Release.