Yes, goats can drink milk, particularly when they are young and need nourishment.
While goats primarily drink their mother's milk (goat's milk) when they are kids, they can also drink cow's milk as a substitute if goat's milk is unavailable. The UC Davis School of Veterinary Medicine recommends feeding baby goats whole cow's milk in place of goat's milk when necessary. It is important to note that other types of milk or formulas are generally not recommended for goat kids.
Once goats mature and are no longer dependent on milk for their primary source of nutrition, they will primarily consume forage such as grass and hay, and drinking water.