Am I the only one that is a little lost here?
You just keep saying diff/differential. Are you talking front, center or rear? There are 3 differentials in your car.
The 04 XT 4EAT was a 4.44 front differential, 4.44 rear differential and of course a 1:1 center diff.
The 06 XT 4EAT was a 4.11 front differential, 4.11 rear differential and of course a 1:1 center diff
You have to match your front and rear ratios if you have a 1:1 center diff, they can be different if you have a different center diff (like out of a WRX).
Now, what diff is supposedly "posi"?
Is the 06 donor from an XT, or a 2.5X/2.5XS?
If you've been driving a 4.11 transmission with a 4.44 rear diff, you've likely smoked your center differential. You'll need a new center diff and a rear diff to match the new transmission.