Can I prune a five year old arbutus?

My five year old arbutus is flourishing, with a good ten inches of new growth this year. I did not stake it until this year, so it leans out. The lower branches touch the ground, and I think the tree would look better if I pruned at least one of those low branches to expose the bark. Is it too late to prune it?