Taiwan is scooter heaven. And, till recently, it was where most bicycles in the world were made. Many, if not most, brands sub-contracted their manufacturing to Giant or other Taiwanese bicycle manufacturers. Even now, the bicycles made in China are often manufactured by Taiwanese companies.
Add to these the fact that Taiwan is a small, densely populated island with temperate climate, and you end up with…not quite bicycle heaven, but it _is_ a very good place for cycling. (For competitive, road-cycling, it may well be heaven!)

Most pertinent to safety is the very presence of scooters. Simply, cars _expect_ you to be on the road.

Public Transportation

The island has a very extensive inter-city bus network, which can take you pretty much everywhere. I’ve had success with carrying a full-size bike in the cargo hold of busses, and certainly a folding bike.

Trains, however, are their usual mixed bag! Depending upon the train type/class/service and the hour, bikes may be catered-to, permitted, restricted, or prohibited. The best bet are the rapid trains with designated cars/coaches for bikes. The High-Speed Train is more expensive, but it caters to bikes very well, if I recall.

Language Barrier

More of a challenge in Taiwan is language.
Though, officially, English is supported by most/all government bodies, banks and utility companies, you really ought to know Mandarin, if not Taiwanese, to explore anywhere beyond the usual tourist traps.
Online maps, and Google Street View can be a great help. But, the romanization of a given street/place name can vary greatly from one reference to another. A street sign will say one thing, an online map will say another! Really, it needs quite a bit of work on your part.

There’ve been expat companies which offered tours and such, but these have come and gone with the years. There are/were lots of blogs, and an online forum or two, but their popularities may have declined with the times.