We did this last June, traveling from Penn Station to Norfolk, VA. Disembarkation was very slow then we had to wait quite a while for a taxi. The journey - in NYC traffic - took close to an hour. Our departure from Penn was at about 1pm (I think) and I would have worried about making an earlier train. We will be doing the same itinerary late May but will make sure we arrange for a driver instead of a taxi this time.