I think the Hero3 battery is slightly smaller, but not that much smaller. Is your Hero3 set to 60fps? That would probably be a drain. Wifi is the other battery drain.
I believe I was using 48 fps and the wifi was turned off (I think). It's too bad they don't allow you to daisy chain as many batteries on top of each other as you want, or at least 3. This would be much better for fixed mount applications. Particularly in a hunting application where you might sit for hours waiting for an animal to come in and then using the wifi to turn a series of cameras on. With only the ability to have one bacpac battery, you can potentially run out of juice before you ever turn the camera on (with wifi enabled).