I have my ANC disabled.
So my thought is this on things. The stock EQ really pushes bass to the 6x9s. More so than a lot of systems. It takes quite a bit of volume for the sub to start becoming noticable over the 6x9s. This is good and bad. With the infinity reference 6x9s in replacement of the factory, the bass response without a sub is really quite good. It's not as hard or low hitting, but it's not bad.
With the doors closed and introducing additional sub volume with the 6x9s and sub all being in phase, you reach a point where the pressures created by the speakers actually begin to work against each other. This would explain why as I increased the volume of the sub, it actually would at higher level begin to reduce the overall volume. As the speaker cones pushed out, they increased the can pressure, and fought against each other.
Think of two people blowing into a pop bottle from opposite ends. They both blow into the bottle, the pressure increases and it's hard move any air. Now have one suck out and they other in and they now work with each other.
That's why I tried to flip the phase on the sub. Sure enough... Now the bass volume continued to increase as I increased the sub volume rather than fading off.
I don't think there is anything wrong actually. Just wondering if my 6x9s are reaching their limits and the overlapping frequencies of the sub and 6x9s below 80hz is problematic. If I crossover the 6x9s (bass blocker) and just allow the sub(s) to handle it (ultimately'in phase'), I may be to overcome that.
Or the other thought is that my 6x9s are actually wired out of phase and the two were cancelling each other.
I thought about replacing the factory amp but that's 11chs I have to feed. That's a lot of extra cost, working, and hassle not to mention dealing with a center channel signal.