In my opionion, goldfish can live their whole lives without ever eating fruit. I have never given fruit to mine. They do get spirulina flakes, peas and occasional zuchinni that is steamed a bit.
They also entertain themselves by eating the algae that I have let grow on the back wall of their tank.