We have had this from a sampling for about 8 years . The tree fruits big almost every year but for as much of the fruit we get the same amount either falls way early or rots or goes kinda brown and doesn't develop or shrivels....
You definitely have a problem with Brown Rot, a common fungal pest of stone fruits. There is plenty of information on the Web about how to control it. Here is a BC government guide that has a section on Brown Rot of stone fruits: https://www2.gov.bc.ca/assets/gov/f...ops/plant-health/gardenguide-13-treefruit.pdf. I just applied a sulphur spray today to control a relatively slight infestation of Brown Rot on my sweet cherry trees. This is the first time that I've tried this; so, I don't know how effective it is. It should be more effective if the spraying is started earlier, preferably around blossom time.