The Treasury has ruled that AHDB's levies are outside of the scope of VAT, a move which could reduce the organisation's buying power by around one-tenth.
The change will take effect from 1 April 2022, with the levy board expressing its 'frustration and disappointment' with the ruling.
The Treasury had notified the AHDB of its formal decision to change its VAT status in October 2020.
But since then, with the support of Defra, the AHDB had been 'arguing hard' against the decision and had asked for a formal review.
While the case continues to be pressed, there was 'no sign of a change of heart' from HMRC, the AHDB explained.
The organisation fears a shortfall of around £4 million on the VAT it will no longer be able to recover on the goods and services it purchases on behalf of levy payers.
According to the AHDB, the organisation's buying power will thus be reduced by approximately 9 percent.
The issue of VAT has been under consideration ever since the levy board was created, back in 2008.
Since that time, it has always charged VAT on levy invoices and recovered input VAT on all goods and services paid for by the levy.
Nicholas Saphir, chairman of the AHDB, said that HMRC's decision would 'punish' levy payers at 'a time of considerable farming need'.
"This decision by HMRC is infuriating for AHDB and will rightly anger our levy payers, particularly at a time of such change and uncertainty being faced by farmers.
“Ever since AHDB was formed we have maintained that we should be entitled to recover VAT and we will continue to fight to try and achieve this."
The AHDB has made an application to HMRC to be included as a Section 33 body entitled to recover some elements of VAT in relation to its non-business activities, which would reduce the impact.
Mr Saphir said: "We also have had the support of ministers in Defra, Scottish government and Welsh government in our bid for Section 33 status.
“Our work towards Section 33 recognition - in parallel with a possible appeal through a tax tribunal - has no guarantee of success.
"In the meantime, HMRC has instructed us to implement its decision and we must comply or face penalties.
“I have no doubt many levy payers will wish to express their feelings directly to ministers at the Treasury.”