Getting rid of remote work will take more than a downturn