I think one of the reasons that people in Europe usually have a passport is because it's so easy to go to a different country...short flight, maybe a short drive and you're in a different country.
In the US, we have 2 choices - Mexico and Canada. For most of our population neither one is a short flight or drive.
Where I live, it's 2-3 hours to Mexico and a couple of days to get to Canada (driving).