Despite there being a sizeable number of armies (we could say codexes) in 40K a lot of the time we find that our armies are facing what should be allied forces. The nature of the game simply assumes that we play and have fun. Nothing wrong with that but a lot of the time for those of us who like a narrative game it can be confusing as to why these armies are bashing heads, so I'm going to take a look at the potential reasons why they might.
Imperium.
As half the game is made up of Imperial armies it can be very frustrating to see two of them matched. There is so many games where you can call "war games" and I'm not even sure the Imperium plays such war games when they have real wars to hone their combat skills. Likewise, if we're playing our chosen Imperial faction you can't say that one side or the other has turned traitor. I know I didn't like that when it came up when I was playing loyalist space marines.
For space marines do we have history of them fighting each other and I don't mean the Horus Heresy. Some chapters have come to blows because of their combat doctrines or points of view (Marines Malevolent, for example). I still think it is unlikely that chapters would start opening fire upon one another just because of this but perhaps the old idea of the Inquisition orders them to could be used but I'd want one side to have an Inquisitorial element to make it work narratively. Again though, like the "war games" idea it happens too often for that excuse to be relevant all the time. There is always some sort of element you could use but I still find loyalist marine chapters fighting going beyond the fluff.
With Blood Angels and their successors you can use the blood rage as a reason for battle. Their Imperial opponent could well be a former now seen as the enemy by a force suddenly succumbing to the black rage. As the 41st millennium approaches it's end and night begins to fall over the Imperium of Man I can see this potentially happening more and more.
Dark Angels are another chapter that could easily go to war against an allied force. While I don't think that in the fluff they have ever openly attacked another Imperial force (prior to the Curse of the Wulfen storyline) I could certainly see it if the chapter felt that their former allies had learnt of the Fallen or even may have been holding one, that they might just decide to kill them and hide the bodies afterwards.
With Grey Knights even I can't think of anything outside of a war game training exercise. They don't turn traitor and there is no reason other than that for their forces to fight one another. They are, however, a perfect opponent for other Imperial forces. If they feel that the other has seen events that mankind (even space marines) should not know then they would have to eradicate them. The same can be said of the Sisters of Battle. If they feel that an Imperial force has been corrupted or turned against the Imperial faith I could see them take up arms in order to put down that heresy.
The Astra Militarum are perhaps the poor ones here. Although they are the majority of the Imperial military they are also somewhat caught in the middle. As I mentioned above I can certainly see many reasons why Grey Knights, Sisters and the Angels turning their guns upon the humble guard army. For the most part though I doubt it would happen but there is a kind of precedent. One of the Dawn of War games, Dark Crusade if I remember right, has a story line where a force of guardsmen are tasked with holding the remains of a downed Titan but this brings them into conflict with the Blood Ravens who see that the task belongs to them. The guard have their orders and aren't willing to relinquish control so the space marines open fire. An extreme example but one that you could use as an excuse as to why your Militarum forces are fighting Imperial allies.
Chaos.
Chaos needs no excuse to fight itself. Ancient rivalries between legions and war bands, or simply rivalries between gods. Even daemons would fight each other simply because their god commands it. I imagine such conflicts go on all the time within the Eye of Terror.
Craftworld Eldar.
We like to think that Craftworld Eldar like each other and work together but let's be honest they probably don't. So why would two different Craftworlds wage war upon one another? The simple answer is that they have competing goals. Perhaps the farseers of Ulthwe read the winds of fate and decide that something needs to be done to draw the Imperium to planet X before the Tyranids get here, but that twist of the threads might ensure that the great devourer passes within range of Craftworld Saim-Hann. I doubt Craftworlds have much diplomacy with one another and I doubt the cares of another Craftworld factor into things. Eldar will fight Eldar.
It may also depend on which Craftworld you play. With the time they spent trapped in the Eye of Terror perhaps another Craftworld feels that Altansar have become corrupted and must be destroyed.
Dark Eldar.
This is another easy one. Rival kabals and wytch cults will fight one another out of sheer rivalry. Perhaps one turned up to ravage an Imperial world only to find a rival there already. The perverse pleasure they may get from dragging their own kind back to the Dark City in chains as slaves or worse could certainly ensure this faction will fight against one another.
But...
Yes, Craftworld and Dark Eldar will fight one another perhaps as much as they occasionally ally. That goes without saying. What you might be pondering is what about the Ynnari? Well, for all the good they are doing for the Eldar race there are those who will be opposed to them. Asdrubael Vect, dark lord of Commorragh certainly hates the Ynnari and I'm sure that there are factions within the Craftworlds who may be opposed to them for one reason or another.
Necrons.
The Necrons are spread across a large number of dynasties and they, being separate "factions" if you will, will fight. They fight for territory whether reclaiming what was once theirs or pushing into a rival's in a grab for power. Perhaps an ancient dynasty never full awoke so various Necron Lord's descent upon it to take what resources they can and repatriate those sleeping warriors to their own cause.
Orks.
Orks are like Chaos, they will happily fight one another regardless of the reason. The Imperium has probably endured so long simply because Orks enjoy fighting each other over working together to conquer the galaxy. Find that rival warboss. Crush his skull and claim his boyz as your own and lead your Waaagh! to glory in the stars.
Tau Empire.
Outside of the conflict between the Tau Empire and the Farsight Enclave I have a tough time working out how this faction could have a inter-faction battle. Not unless you are willing to see a Tau civil war, although I tend to see that as the Tau / Farsight conflict anyway, so you could always use that. Not like anyone is going to know outside of yourself. You could perhaps have a break-away group following the recent assassination of Aun'va which needs to be forcefully put down or re-integrated into the Greater Good.
Tyranids.
This is the one where quite frankly I have no idea. There is absolutely no way in my mind as to why two hive fleets would fight one another. They are the same controlling mind so it just would not happen. If one hive fleet was dying or too weak, it would happily just roll over to be devoured by another, ensuring all DNA and enhancements were given over to the hive mind. They just would not fight. Even genestealer cults would give up once their "gods" arrived.
However, saying that, many years ago I did run a campaign set on a world where ancient Eldar had hidden hundreds of thousands of trapped daemons deep under ground and I hade to include something to allow the possibility of multiple opponents of the same faction. As those daemons began to break the prisons it released a powerful psychic shriek which disrupted the telepathic communications of any hive fleets in the area forcing them to become separate from the hive mind. Multiple tyranids could then fight as rivals seeking to restore bio-matter in a sort of hive war.
So you can do it but you need to come up with something good to explain why two separate hive fleets would come together to fight one another.
