Solved :) It turns out that if you run an application pool with a custom identity you need to set individual permissions on the metabase keys using metabase explorer. I had already added the IIS_WPG group that the account belongs to, but this doesn;t seem to be enough. Adding the actual local account...