I feel its a little biased. The husband went back to the past and chose a different wife but the memories (be it good or bad) of the previous wife are still there therefore he have a deeper feeling for her as compared to the new wife cause he have no memories with the new wife at all except their high school concert.
Even though conflict and unhappiness is normal in a marriage but there is absolutely no valid reason for the previous wife to treat the husband so terribly.