I finally heard back from the Vietnam e-visa people. Their response is copied below. It appears that if Phu My is your entry port to Vietnam, an e-visa will not work. Hope this helps clarify the issue.
It seems that you will dock at Phu My port as soon as you arrive in Vietnam. However, Phu My port is not on the list of E-visa. (Phu My port is different from HCM seaport (called Sai Gon seaport).
In this case, you should obtain a visa from Vietnam Embassy nearby.