Just want to thank both crossan007 and riro for this solution!
Even though my log stated that a lot of drivers had failed to migrate, it was all due to just three old deleted categories.
And for any powershell/wmi noobs like me. I needed to change $Namespace to the namespace of my sccm site, i.e: "ROOT\SMS\siteP01"