Sometimes eBay's API tells Pirate Ship that an order still needs to be shipped, even if it's already been shipped or has been cancelled.
You can use the "Ignore Order" option in the Action menu next to the order to hide these orders in Pirate Ship 👍
If you'd like to solve it permanently, eBay's customer support says this is the solution:
In eBay, go into the sale record. Make a note of the tracking number.
Use the dropdown menu on the top right to "Mark not shipped."
Do it again and mark as shipped, with the tracking number you used. That should reset the data Pirate Ship gets from eBay.
In Pirate Ship, click the "Refresh Data" button in the Import page and the order should be gone now.
If you have any issues, please contact eBay's support team, as they're the ones who have control over this and have told us these instructions 😉