Ugh, yea it's so hard to fly cross country with the big time zone changes. When we visited my parents in New York in August, the only direct fight back to Phoenix left around 6pm, and we landed at 9:30pm local time, which felt like 12:30am NY time, and by the time we got home, it was close to 2am NY time. That was brutal!
Even still, I think you are making the right call with keeping the later flight. It's not worth the risk of missing the flight since based on peoples' replies, that traffic nightmare leaving the port of Long Beach is very common.