RadSpell should use the MaxJsonLength value specified in the web.config
You can find more information about the problem, cause and repro in the following thread: http://www.telerik.com/community/forums/spell-check-hander-error-the-length-of-the-string-exceeds-the-value-set-on-the-maxjsonlength-property