We inherited 3 lovely Nerium oleander shrubs grown in pots when we moved to Nanoose Bay - not usually as hospitable to Zone 8+ plants as Victoria. The previous owners moved the pots to an unheated but sheltered area under the deck for the winter but we, unfortunately, left them out in the elements and they died. From this experience, I would say you could definitely grow oleander in pots (taken in in winter) if not directly in the ground.