Personally, the only problem I would have with joining EFTA is how long it would take our useless politicians to change their minds and start the Leave process all over again. Labour because there would still be too many rules holding them back from pillaging business and individual wealth to fund their spending plans. The Conservatives because they still want to free the UK of all EU shackles. EFTA would solve the Irish border problem, keep freedom of movement and get that all important ease of trade. The EU would still demand its pound of flesh for access to the single market - if we are lucky 5 billion quid a year - but we would be free to farm and fish to our heart's content. No individual trade deals but it would lock us into the existing free trade deals and some kind of fudge would be needed for the UK to play a role in future trading deals via the EU.
Immigration was one of the big reasons to leave, FOM would have many people screaming their heads off but my much mentioned 5-10 year retrospective residence test, with foreigners moved out of social housing, before any access to welfare, etc., would balance that out.
Whether the existing EFTA countries would let us join is another matter, though having the weight of the UK behind them would make their position more powerful in future negotiations with the EU. Swings and roundabouts.