According to the Institute for Fiscal Studies (IFS), male workers with fewer or no qualifications are likely to be hardest hit work wise.
A government spokesperson said it had proposed frictionless trade between the UK and the EU.
“We have put forward an ambitious proposal that will allow frictionless trade between the UK and the EU, protect jobs across all sectors and deliver a successful Brexit that works in the interests of the whole UK,” the spokesperson said.
“It is in no one’s interests for there to be trade barriers between the UK and the EU which is why we have proposed a UK-EU free trade area underpinned by a common rulebook for goods.”
The UK currently has extensive trade links with the EU, with 44% of UK exports destined for the bloc, and more than half of imports, the IFS said.
The most vulnerable to increased trade barriers after Brexit are men who work mainly in factories such as car manufacturing, clothing, chemicals, and pharmaceuticals factories who have GCSE qualifications or below.
These industries sell a large amount of products to the EU. The car industry is particularly at risk, because it also imports parts used in vehicle manufacture from the bloc.
Approximately 3.7 million people work in industries the IFS regards as “highly exposed”, meaning they would lose more than 5% of their value if trade was under WTO rules after Brexit.
There are also “very highly exposed industries” that employ about 500,000 process, plant and machine operatives.
These people “may find it particularly hard to move into less exposed industries as workers in this group tend to be older and less likely to have formal qualifications than other workers”, said IFS research economist Agnes Norris Keiller.
However, other sectors are also at risk, the IFS said. The financial services sector is particularly exposed, as about 12% of its business is with the EU.
“Crucial here will be the scale of non-tariff barriers, including the potential loss of passporting rights, after Brexit,” the IFS said.
Data obtained by the BBC showed that populations with lower qualifications were significantly more likely to vote Leave.