From what you've written, it sounds like the Explorer II is more appealing to you in every way except concerns about its size.
I've never tried one on, but I do have a 42mm Tag as well as a sub c and any size difference between those is not noticeable to me. I agree that the sub c "wears big" for a 40mm watch.
Is it wise to swap? I wouldn't have thought so financially, but then wisdom often goes out the door when it comes to watch purchases!