Yes, the Phantom is VERY undersized... The Ghost is a pretty huge ship itself. I'd say a good deal bulkier than the Falcon perhaps (just eyeballing things without either in front of me to be sure). Keep in mind The Ghost is dual-level... Not sure if they're much different in diameter, but I'd say The Ghost is probably larger than the Falcon if I were to make a guess on it.
Which would make sense, with it having essentially a little "runabout" shuttle that can dock to it... The Falcon doesn't have the bulk for something like that.
The Ghost is very similar in style, but larger than the hero ship that was in X-Wing: Alliance, which is significantly smaller than the Falcon. All YT series Corellian ships (The Ghost is too, iirc).
The Phantom is very akin to a shuttlecraft in Star Trek in that it has a small back cargo area and a relatively spacious cockpit capable of seating at least 2 up front and a 3rd behind them in a sort of jumpseat. It's no tiny thing.
Hasbro's always managed to fudge designs to work for stuff like that, so I think a dramatically undersized Ghost is doable that would still dock The Phantom nicely without looking too goofy. I think the play features would be where it's all at. It wouldn't be high on my radar (I passed on The Phantom already) but I think it'd be a decent toy if done right... Now, if they cut construction and design corners ala the mid-size vehicles now, Eh, it could look (and feel) a lot more "cheap".