I LOVE Harry Potter and have gone to Universal for that reason. I honestly think its doable but go in with some realistic expectations. The walk into the park from bus drop off/uber/parking is very long. It is another considerable walk to the areas in each park that are Harry Potter themed. If this is the only time you ever get there then you might find it worth while but you will not get everything done even in the Harry Potter portions. If you could swing this as a different trip it might be better saved.
On another note, if you decide to do it. The train ride between the two par