There's some pretty simple rules of thumb when it comes to knowing what will be the cheapest way to ship:
If your package is 15.9 ounces or less, then using your own packaging with USPS Ground Advantage service will always be the cheapest option.
If your package is over 1 pound but it fits into any Flat Rate Envelope or the Small Flat Rate Box, those are the cheapest ways to go.
If your package is over 1 pound and doesn't fit into those, then it's cheapest to use USPS Ground Advantage, as long as you're okay with a 2-5 day delivery speed.
If you need a faster service, use as small of a box as possible to qualify for Priority Mail Cubic.
Only if your package weighs more than 20 pounds or you're shipping to the farthest zones and it weighs more than 2-3 pounds would Medium or Large Flat Rate Boxes ever be a good deal.
One benefit of Priority Mail Cubic is you can use your own packaging, but USPS also offers free boxes you can use with Priority Mail Cubic:
Priority Mail Box - 1096L | OD: 9-7/16" x 6-7/16" x 2-3/16" | Cubic Tier 1
Priority Mail Box - 4 | OD: 7-1/4" x 7-1/4" x 6-1/2" | Cubic Tier 2
Priority Mail Box - 1097 | OD: 11.625" x 2.5" x 13.4375" | Cubic Tier 3
Priority Mail Box - 1092 | OD: 12.25" x 2.875" x 13.6875" | Cubic Tier 3
Priority Mail Box - 1095 | OD: 12.5" x 3.125 x 15.375" | Cubic Tier 4
Priority Mail Box - Shoe Box | OD: 7-3/8" x 5-1/4" x 14-7/8" | Cubic Tier 4