you'd be more than welcome at my mothers, but unfortunately they too have moved on, and now live in Swaziland.
Port Elizabeth has a few bike shops, but the biggest and the best is Sharwoods down off the main street, (think its now called Govan Mbeki Drive) They stock just about everything you could need. I dont think its a wise idea to travel without insurance, but as far as I know, you would'nt be breaking any law that I am aware of. Perhaps it would be best to contact the SA Embassy regarding insurance. Although I am sure that you can get away with it if you so choose. The best place to get a bike in Port Elizabeth is in the classified section of the local paper, and the biggest choice is always on the weekend. (Its called the Weekend Post- and it features wonderful stories about old people who knitted jumpers and pictures of kids who can run fast for their schools--i.e a thouragly shite read!)
Depending on what you want to buy, I would think that PE will have what you need. There is not a great deal of second hand bikes, and it may be more worth your while to go to Johannesburg to have a better selection (and they wont be rusted like the local bikes)
hope that helps