Done this many times. Maine is usually faster and less busy. As long as you don't mind crossing the border, I highly recommend the "airport route," as I have heard it called, through Maine and back through St. Stephen.
Also, Ganong chocolate has their factory and outlet store in St. Stephen...always a good stop after waiting at the border.
GO HABS GO!