When images are added to the document with non-default quality (PdfExportSettings.ImageQuality different that ImageQuality.High) , they are re-encoded, which uses a lot of memory and may cause OutOfMemoryException for large images.
Workaround (works when inserting big JPEG images): In this case ImageQuality.High may be used and RadPdfProcessing will decode only the image headers to insert the image into the PDF file. As image pixels are not decoded with ImageQuality.High, there will be no OutOfMemoryException.