The novels primary purpose was to help sell the game, mostly in its table top form.
From a game sales standpoint a new era ment new mechs, vehicles, aerospace fighters ect to be sold as miniatures, and technical readouts. Plus it allowed them to add new factions in the form of the clans.
From the lore standpoint the big jump was partly because the lore had placed itself into a bind - the union of the Lyran Commonwealth (most dynamic economy) to the Federated Suns (largest most powerful Inner Sphere military) ment that sooner or later the new Federated Commonwealth would conquer the Inner Sphere. Plus they had Superior Leader Hanse Davion.
Really all that they were left with was either a superior leader in one of the 3 remaining factions being able to reverse the expansions of the formidable Super House that has been created in the Steiner-Davion axis, or the new Super House civil warring against itself, or someone stronger showing up to knock them off their high horse.
They chose 3, 1, and 2 in that order. (The Clans, Sun Tzu Liao, and Victor vs Kathrine - the Grudge match of the 31st Century!).
As for events before any of the novels proper, no, many of the things you mentioned could have been good places to build stories around, in fact Battletech has a hugely rich lore with hundreds of places coherent book plots could be written from. Off the top of my head:
The founding of the Star League
The machinations of Stafan Aremis resulting in the End of the Star League.
A novel following the life of Alexandr Kerensky
A novel following the Cameron family dynasty.
A novel detailing the political compromise that led to the creation of Wolfs Dragoons, and the early years of that famed unit.
Something about the first succession war (you go from a people who really hadn't seen war in generations to a people who can't kill each other fast enough - would everybody buy in, or would there have been attempts to end the slaughter?)
The Marik Civil War.
Any number of infamous pirate/merc units.
Ect.
Edited by Tyrnea Smurf, 24 June 2015 - 02:29 PM.