To reproduce: run the attached sample project, open the popup and try to to scroll the grid with using the mouse wheel. You will notice that the popup is immediately closed.

Workaround: 
    Private Sub PopupClosing(sender As Object, args As Telerik.WinControls.UI.RadPopupClosingEventArgs)
        If args.CloseReason = Telerik.WinControls.UI.RadPopupCloseReason.Mouse Then
            args.Cancel = True
        End If
    End Sub