Huge crowds travelled to Thorney, near Langport, on Sunday for the annual celebration which features fundraising competitions including a raft race and mud wrestling, and organisers expected to raise a record tally for local charities.
But in the early hours of Wednesday morning, vandals used a digger to rip down the main marquee.
They also set fire to a caravan on the site and smashed up a vintage lorry that was awaiting collection. A nearby barn owned by a committee member was also set alight at around 3am.
Committee chairman Andrew Wakely said the attack could have a serious impact on the contribution the Lowland Games is able to make to charity. The cash is still rolling in, and Mr Wakely had hoped to top 2008's record tally of £8,000.
Charity contributions are made after the costs of the Lowland Games are covered, and this week's attack could have a serious impact on donations to be made to organisations including the premature baby unit at Musgrove Hospital and St Margaret's Hospice.
Mr Wakely said: "We are in talks with our insurers so we don't really know how bad it will be yet. It's too early to say whether the attack was specifically targeted at the Lowland Games but it is a very big coincidence that a committee member had a barn set on fire.
"It is a real pity because the games were fantastic - the best we have had in a long time. We had a lot of new family attractions, some great music and very good weather."
Anyone with information relating to the Lowland Games vandal attack or the barn fire should contact Crimestoppers on 0800 555 111. Turn to page 8 for more pictures.