Greetings and welcome to the board!
For goldfish, sand is generally a big no no. When
they are foraging through the bottom, they suck up
stuff which could be passed through their gills. Sand
will tear up the soft tissue of the gills.

It would
be similar to using sand paper on your tongue.