interchangeable?

Yes

I don't think you can put #4 gears, shafts and bearings in a #5 case or vise versa, can you? I think to do the swap, the lower would need to be complete with the correct internals in it.

Complete will bolt up.

Good question and I'd like to know the answer as well. As for a complete lower swap I believe one potential issue could be the polygon vertical shaft in a SSM IV, not sure the V used a polygon shaft.

So what is the weak area of the #4? The upper or lower? I know that Gary sells straight cut gears for the #4 that will make it equal to, or close to as strong as a #5.

A SSM4 is a shortened SSM3. A SSM5 is a shortened SSM3A. I believe the main difference is 3A and 5 have bigger shafts.

The gears are better in the 5 vs 4 and the bearings are bigger as well. Other than the prop shaft, I don't think there is much difference in the actual shaft size in 4 vs 5. They are cut different. Splined (5) vs polygon (4), but as far as I know, the overall diameter is the same.

Finally got a hold of Collabella ,.. the 2 are not interchangeable.
The 5 lower has bigger stuff in it..

That makes sense 'cause I researched putting 5 lower intervals in a 4 housing and that wasn't an option.