As this website has a page on sheep sculpture, I thought that a suitable precedent has been set for a modest page on goat sculpture.
The goat is not particularly prolific in British sculpture, but is found from time to time in heraldic imagery, and symbolically in scenes representing wilderness or wildness, or wild abundance, as opposed to the decorous and tamed natural abundance indicated by the sheep as an allegory for pastoral or agricultural themes. Thus the picture at the top of this page, a rather deco sculpture, has at the left hand side a field of corn behind the man, in the centre, the woman (herself an allegory of fruitfulness) holds a bowl of figs or pears, indicative of fruit collected from the wild, and to the right, a goat on a stony base indicative of wilderness. The animal's gaze is focussed on the bowl of fruit, suggesting perhaps an untameable, unquenchable wild lust for life.
Goats with putti or infants are sometimes found, which may tend towards the gentle, as in the picture below left, of an infant boy with a goat kid, or the more animalistic untamedness indicated below right, where some young Cupid wrestles with a truculent he-goat. This one is by Albert Hodge, and is one of several slightly mythologically themed animal groups by that sculptor.
Taking our wild abundance idea further, we have the goat as a pagan symbol of virility, which of course comes to its classical fruition in the idea of the Faun, half man, half goat. The example below left, an early 19th Century panel following an antique theme, has the figure of a Faun on the left, holding pan-pipes and pot of wine, gazing appreciatively at a Bacchante dancing to the tune of a pipe. More aggresively, the group below right, a late 19th Century work again based on a common Classical theme from Attic pottery and small bronzes, depicts a Faun lustfully seizing a naked dryad by hair and neck. The man with goat's horns but human legs is generally based on the Greek satyr, familiar from ancient pottery - here is a page on satyrs.
More peacefully, the goat can simply evoke something wild but unthreatening creature, as in the group below left, of a rearing goat and a kid, in a pyramidal composition, an appropriate decoration for a Victorian orangery. Even more decorative is the goat medallion in the Natural History Museum shown below right, where the roundel is echoed by the sweeping curve of the rather Ibex-like goat.
Most peacefully of all, we have the kid-goat, small and weak and trusting, held by the Goatherd's Daughter, which stands in St John's Lodge Gardens in Regent's Park, London. The sculpture of this charming group is C. L. Hartwell.
The goat's head is difficult to be sure of - are these examples below goat or sheep? They seem to have no symbolical meaning, but are used for decorative effect only. The one on the left has very curly horns, but is so mild of expression that perhaps a sheep is indicated, while the right hand example, from a short, composite pillar, seems wilder and more goatlike.
We end with an example of the goat in heraldry, from a coat of arms on the side of a decorative lamp-holder in Cardiff. Here we have the goat rampant, perhaps indicative of masculine virility, as the hippocampus on the other side of the shield suggests seaborne exploration and intrepidness.
Heraldic goat rampant. 
Visits to this page from 13 Mar 2014: 6,926
Sheep sculpture // Other animal sculpture // Allegorical sculpture
Man with goat's horns - satyr sculpture
Fauns - half man, half goat. 
Decorative goat sculpture.