This is in zone 4/5 (Collingwood ON), an escape of some kind. It looks like some kind of prunus to me (false thorns, color of bark, fruit). The only thing foreign to me are the skinny snaky side branches. The fruit has a long stem and grows from the skinny side branches. The fruit has a round pitlike stone similar to a sweet black cherry but it seems to be have been less fleshy and the birds did leave at least some of them on the tree.
Flowering quince? Bit of a long shot given the description of the fruit I suppose. But the spikes are evocative.
