Spoiler: the issue is in the css-loader update and the fix is outlined here. The css-loader needs the esModule: false option. My final webpack.config.js looks like this:

More on the issue

I was updating an old-ish (3 years) Vue app which had been updated but not regularly. I got everything working again as you do but couldn’t get the styles to load. It took a lot of searching to find the issue was in the css-loader package which now has a default of esModule: false. This is described here in their docs and was a breaking change not compatible with vue-loader.

The bit of my old webpack config that handled loading looked like the following but updating and setting esModule: true as in the snippet above has worked for me.