Port Townsend has an extensive network of public trails. Many of these were platted as roads but never developed. For the most part, you can get from one end of town to the other without having to ride or walk on busy streets. For a map of possible walking routes within the city limits, check out the PT Walking Map page.
There is also the Larry Scott Trail, which goes from Boat Haven to the edge of town, and beyond. It's maintained for use by bikes, walkers, and horses.