Johnny91 I don't really know how to answer your question, but I personally wouldn't consider 5-20% CPU usage a lot. I would consider that a good vital sign for the apps running on the server, and it leaves lots of margin for spikes, additional apps and a higher volume of visitors. Any server will always require a baseline CPU usage anyway, just for handling incoming connections, operating disks and running internal tasks.
It all depends on what apps you are running and how many visitors you have, but I don't quite see why you think it is too much ...
Also, you mentioned you switched from NVME to SSD, but did not explain if you saw any difference in CPU usage after switching. I would not expect changing storage would affect the CPU, but I could be wrong.