It will be hard for someone, most likely. However, at this juncture we have no idea who that will be.
My purchases from Germany have been comparable to UK, save Hattons who subsidise postage.
As I previously stated, businesses like Hattons have been deducting VAT for customers outside the EU for years. If they want to sell to people in the EU, then businesses would be stupid not to offer the same facility to the EU after Brexit. Simple good business sense.
I have bought from the States regularly and if you go over the €150 duty (not VAT as you correctly pointed out, my bad) the penalty is large. But I still save over buying the same product in the EU.
We simply have no idea how Sterling will react, but economists predict it will drop and already has since the Brexit vote.
I guess I'm more a 'glass half full' kinda guy.