The Classic RenderMode displays the Crop icon for custom tools, the Lightweight RenderMode displays a gear icon

Workaround:

			/*set background image*/
			.rieSomeCommandName
			{
				background-image: url(my-images/icon_16x16.png) !important;
			}
			/*add for lightweight render mode*/
			.rieSomeCommandName.rtbImage::before
			{
				/*set your own font and/or content if you want to use a font icon instead of a raster image*/
				content: "" !important;
			}

where the command name is taken from the tool by concatenating rie + <command name>:

			<telerik:RadImageEditor runat="server" ID="RadImageEditor1">
				<Tools>
					<telerik:ImageEditorToolGroup> 
						<telerik:ImageEditorTool Text="some text" CommandName="SomeCommandName"
							ImageUrl="my-images/icon_16x16.png"></telerik:ImageEditorTool>
					</telerik:ImageEditorToolGroup>
				</Tools>
			</telerik:RadImageEditor>