You'll likely get more feedback if you are more specific in the countries or region of the world you are loosely considering. One would assume you are mostly focussed on Europe but the Aussie reference confuses it a bit.
In general, you can buy and register a bike in almost any country. They will usually sell to any temporary resident and in some cases tourists, and in many countries you can claim to be a temporary resident simply by providing any temporary address. In others, you actually have to provide some backup but in most cases they are happy to take your money and have you pay taxes.
The main limitations are finding a country with low bike prices and low taxes minimizing total costs. It also helps to buy in a country with a fairly straighforward registration process. It also helps to speak the language.
There are a couple of old threads on HU about the best place in Europe to buy a bike which you can search if you want to. I would suggest UK and Germany as two places to start. on the other side of the globe, Oz is ok, but prices are in general higher.