What USDA Cold Hardiness Zone are you in? In the colder zones, they die back to or close to the ground. You should probably prune it back to reduce damage by winter snow and wind, but leave enough of the canes. If there is winter kill to the canes, these can be pruned back to healthy tissue in the spring. I would also assume that you have the more common Buddleja davidii, which flowers on current season's growth. If you have Buddleja alternifolia, you would leave any pruning to immediately after the flowering is done - in spring - since the flowers are borne on previous year's wood. Pruning Buddleja alternifolia in the winter will result in no or poor blooms the following year. If you are in a colder zone, Buddleja alternifolia might actually be a more suitable plant, as it is more cold hardy and has been reported to survive in USDA Zone 5. That is, if you are lucky enough to find a source for it.