While this task could be accomplished by visiting the BIOS settings, it could be a lot of hassle. Eccentrically, Windows does not have any in-built tools that could let you monitor your CPU temperature.
Does your PC feel sluggish, start slowing down while performing some intensive tasks such as video editing or gaming? Then overheating the CPU and its cores may be the culprit behind it.