If you make this with apricots, I recommend skipping the lemon juice. I find the skin of the fruit gets unpleasantly tart otherwise.
The number of plums you need depends entirely on the size of both the fruit and the pan. I recommend buying more than you think you need because you'll almost certainly want to make a second cake.