Can Golden Sun 4 support a vs mode?

It'll never happen, for the simple reason that it's simply not cost effective to design two entirely different battle systems, one of which won't even be used in the main game. This especially holds true if the series remains on handhelds.
Unless I'm misunderstanding what you mean by a system like Smash Bros.

The usability of characters from the previous games doesn't really sound all that great either, because characters like Matthew and Tyrell are just carbon copies of Isaac and Garet right down to their movesets so there isn't much point beyond appealing to people who like the original cast more.

Playing as villains would be kind of neat, but their moves and stats would have to be scaled so that they were beatable in a one vs one environment that you're advocating.

I was thinking of the vs mode being a completely different element to the Golden Sun series.

Maybe Super Smash Bros wasn't a good example.

Maybe a Dissidia like experience where we still pick a character and battle another character?

Then we could customly set up to 4 psynergy attacks per character.

We could also equip a weapon obtained in the regular game for each character.

This mode has its own level up system, although characters in the regular game can gain levels within the regular playthrough.

There would be characters unlocked in a catalog.

It would be all main protagonists and antagonists of every Golden Sun game made.

Also if you don't have someone to play against, no problem just choose a cpu opponent and fight it.

So what do you think guys, could this idea work?

It'd work better as it's own game, and I'd prefer it to be more like Super Smash Bros. rather than Dissidia, tbh.