I rode London to Astrakhan last year - via Baltic countries and Russia. Effortless riding, generally highly interesting and enjoyable except for the long distances.
Something in the back of my mind makes me think the Russia/Azerbadjan border is closed for foreigners these days. If so, it would be because of tensions and security concerns around Chechnia, Dagestan and other wannabe breakaway areas. Rus/Georgia border is closed to everyone - except invading armies!
Travelling on a non-EU passport will probably require numerous Visas depending on your Passport. e.g. UK, Schengen, Russia, and depending on your route Belarus, Ukraine and more, just to reach the Caspian. Then there's Kazakstan, Iran, Iraq.
Not unsurmountable - just a pain.
Roll on the day of no Visas - my New Year's wish!!