Thanks for all the feedback guys! Yeah I've got over 10 years in construction on the management side, plus a couple more as a grunt.
I agree with hobomaster 100% that subing out the dirty work makes the most sense. That's why I mentioned drywall taping and mudding would be a sub, while drywall hanging would be me.
I think the other thing that'll make a big difference is going with a pre-built house structure. I'd like to eliminate as many middle steps as possible, and have a sub for the foundation, and then some subs to help me finish the interior work.
Luckily I have access to a bunch of bored engineers in the winter, so they could help me put together a schedule and estimate based on the plans! I could do it myself but why not use their experience to help.
One last ace up my sleeve... A retired dad who built his own cottage by hand (literally he had no power, cut all the boards with a hand saw, hammered every nail by hand!). I'm sure the bank wouldn't want it to drag on into years, so having him available (I could fly him out and take a couple weeks off to work with him) could help keep things on schedule.
I managed to get things of Kijiji when I was building my sauna, and I completed it for about 1/3rd the cost of a kit, so I'm sure I could look on Kijiji again for other oddball things when the time came (tiles, heavy wood front door, etc.)