The problems with sand as a substrate for goldies is that they are bottom feeders. They root around at the bottom of the tank for hours looking for something interesting to eat. Not only will they constantly stir up the sand in this activity - which could lead to a constant haze in your tank instead of clear water - but in doing this they take the sand into their mouths and it passes out though their gills. Sand, constantly passing over the delicate tissue of the gills is an irritant in the least, and can cause open wounds if a sharp piece gets caught. It is like having sand paper rubbed over the gills.
Because of this, sand is not recommended for a goldfish tank.
If you use very little gravel, be aware that they will also take it in their mouths. If it does not pass out the gills, then they could swallow it.
I, personally, feel that the best gravel for the bottom of a goldfish tank is a thin layer of large enough stone that it does not fit in the mouth of the biggest fish in the tank.