The performance difference can be noticed from 75F to 90F, but you'd have to be running past 100F (+110F on the tarmac), to start noticing some real bogging.
But yeah, I think eliminating the cat would do more in terms of power.
Eliminating the CAT on our cars does not give significant HP increases in fact they are negligible. It also just causes more pollution, Congrats on suggesting contributing global warming. Lol
If one wants more flow they can run a a high flow catted downpipe. Yet unless one has h upgraded their Turbo, a high flow catted downpipe is also unnecessary. The OP had not upgraded his turbo so he has no need whatsoever to change his downpipe. He was asking about intercoolers not downpipes.
On the Intercooler front indeed our stock intercoolers are wimpy and if you live in warmer climes you will witness heatsoak. Which will cause a loss in power overall.
All it takes is a couple of hot laps and the car will need to cool off before going again on stock intercooler.
The other weak link like others have mentioned and lots of us in warmer climes have experienced is the radiator it can and will overheat under prolonged use in 90-100 degree weather with the Air on. Also in stop and go traffic .
A lot of us have changed out our radiators for a Mountune or Mishimoto the two most available brands. There are a few others out there but they are either no name ebay Chinese knockoffs or lesser known UK brands.