CDC's highest travel risk level is jam-packed with nearly 135 places, including Japan
https://edition.cnn.com/travel/article/cdc-very-high-risk-travel-destinations-february-7/index.html
Death rate or death toll should be added to factors to decide risk level. If CDC takes infection numbers into consideration, countries unwilling to carry out PCR inspection would be underestimated.
In the first place, just visiting somewhere has nothing to do with infection. Higher risk lies in the situation that people are talking in densed area without masks. Only if you avoid it, your risk of infection doesn't change.