You are required to have registration in Australia and if you don't, you will very likely get caught unless you are staying in the outback only (and even then...). The police have scanners which read number plates and run them against a database, and if you're not registered they'll pull you over. Registration in any state or territory allows you to drive in all Australian states and territories.
Compulsory Third Party (CTP) is the only compulsory insurance. It will only cover you against causing injury/death to OTHER people (eg road users, pedestrians). In practice you pay this with your registration costs. All insurers are by law required to charge the same amount, but you can choose which insurer to use.
Of course you can get additional insurance separately. You can get Comprehensive which insures against fire, theft, or damage to your bike (and sometimes your riding gear) even if you cause an accident, and will also insure against damage to other vehicles/buildings. It is the most expensive.
Or you can get Fire and Theft which of course insures your bike against fire and theft. But you are not covered if you cause damage to other vehicles, buildings etc in an accident. If another driver/rider causes an accident and causes damage to your bike, they are obliged to pay for repairs (their insurance will usually do it if they're comprehensively insured).
All insurers are required to have a product disclosure statement (PDS) which is the legally binding agreement/contract. They will tell you what you're covered for and what you're not covered for, your obligations, and other details.
As far as insurers go most of them in Australia are OK but I also use InsureMyRide - they have been inexpensive and were easy to deal with when I had to make a claim. Five stars!