To reproduce: please refer to the attached sample project. Click the button several times to add 3-4000 rows and toggle the checkbox. You will notice that every time you add new rows and hide the column it takes a long time. The more rows you have, the more time it requires.
Workaround:
radGridView1.MasterTemplate.BeginUpdate();
radGridView1.Columns[0].IsVisible = checkBox1.Checked;
radGridView1.MasterTemplate.EndUpdate();