Tax buildings often have the side effect of increased County Growth, which at low levels doesn't make much difference, but by 50/100 Development can net you an extra 25% Tax and Levy Size for that Holding. All rulers need to tax their people and gold is hugely valuable, especially in times of war when your raised armies suddenly suck you into a black hole of debt which cripples your fighting ability. The third priority, and probably worth dropping some Defender Advantage for once your position is more secure is Tax. This also applies to other held Counties where you can have those same two buildings and receive the same buff a second time. As we mentioned above, you can stack bonuses so if you have two buildings that buff archers, both will be counted. These will add large numbers to your Levy while also boosting the damage and toughness of archers and skirmishers which make up the cheaper of your Men-at-Arms and as such will be the early to mid game units of choice. Many buildings that increase your Levy also offer Realm-wide bonuses to the Men-at-Arms units that will be your heavy hitters. As with most things, it comes down to numbers and being able to marshal even an extra few hundred men early game makes a huge difference in who you can defeat. Your next step should be increasing your personal Levies. Coincidentally some of these also happen to be the cheapest options which make them even better choices for the early game.
When fighting a defence battle these roll modifiers can turn defeat into victory, but if you were to only choose one, the Bailey offers much better benefits and remains viable well into late game.
A good starting point may be to build a few Defender Advantage buildings such as the Outpost and Bailey. What you choose will be dependent on what you need most at the time, and all buildings can be replaced at a later date, although you do lose the initial investment.