I think you've figured this one out. Like the DR650, the DR350 does not really need the lower roller, especially for mostly road travel. I've ditched both upper and lower on my DR650 because they made so much noise. No chain derailments in 30K miles and two chains/sprocket sets. Quite a bit of off road use with luggage and bottoming. No worries so far!
As long as your chains/sprockets are lasting well I think you are fine. To me, this is key.
I would look along the chain path: swing arm, rubber swing arm slider, frame rails, chain guide and the area around countershaft sprocket. Make sure the chain is not abrading metal any where or rubbing where it shouldn't.
Many neglect the swing arm rubber Slider and let it eat up the swingarm. Replace if worn out.
The other thing to check is the rear Chain Guide, the plastic thing just ahead of the rear sprocket. These wear out eventually. (I assume the DR350 has one?)
As has been said, the upper and lower rollers are mostly there for extreme off road use in big whoops where suspension is repeatedly bottomed. In these condition your chain would need to be pretty loose, and without the rollers it could skip a tooth or de-rail, but unlikely. Keep the chain adjusted correctly for road riding, you should be fine.
Just keep an eye out but I would say NO ... You do not need either one. Maybe check out a DR350 board to verify, get DR350 owners opinions?