I was just thinking about places in Australia that would be worth a visit, and that I have been to and would like to visit again. Green Island is one option.
As I recall, I went there in December 2009 and it was an amazing place. Unfortunately I was a bit sick at the time and not able to do scuba diving due to an ear infection, but this is definitely a worthy candidate for a 4 day break. The beach is very peaceful and clean and there is a certain tranquility to the place. Sitting on the edge of the water letting the warmth run up over your legs is a memory that will live with me for a very long time.
I think that it helped that the holiday there last time was building towards Green Island as we drove up from Sydney (I can't recall how far, but maybe 3,000km). Plenty of nice places along the way as well, such as Byron Bay (and I nearly got a tattoo there!)
Airlie Beach (although there was little swimming there as "stingers" were in season)
Another place that was nice, but understated was called Clareview.
A small village that you would miss if you blinked, but had something about it. The sea was a long way out and it took about 10 minutes to reach once you hit the beach! We also caught a crab (but put it back much to the disappointment of my girlfriend who wanted a new pet!) If you ever happen to drive to Cairns, stop there!
I also have not been to Ayres Rock before, but would like to.
A distinct lack of beaches there (although plenty of sand I would guess), but it will be warm!
Anyway, that's it for now.