I have researched this behavior and it is related to the popup control, not to the ListView.
The interaction with the popup is as follow, when the popup is displayed and it is non modal, you have to tap outside the popup to be closed and then you can interact with the elements on the screen. In your case the PullToRefresh gesture. So we have the following feature request to provide an option to interact with the content outside of the popup when popup is open: https://feedback.telerik.com/maui/1634965-popup-provide-a-way-not-to-close-the-popup-when-the-user-clicks-outside
I am setting this item status to Duplicated as it is related to the popup feature request.