I left my delete out for the winter as well.
I will eventually double sided tape some sound studio-like acoustic 'egg crate' foam back there to cut the booming without adding any significant weight back into the car.
(Or a piece of .75" thick closed cell foam, like a yoga/exercise mat or such, if I cannot find the noise cancelling/absorbing foam anywhere for a decent price.)