I'm in the same boat MB, i took a long break from the franchise after halo 5. The possibility of Infinite being great has brought me back. And the fact that its coming to PC has gotten me excited.
I felt that one of the main problems with the new direction was actually 343i playing it too safe and going with fanservice instead of sticking to their guns. Especially in halo 4. I always thought the covenant storm, the ur-didact being a villain, the prometheans, the banished, and even the Cortana AI overlord plot, were all results of them playing it too safe. Instead of introducing something new to the series, such a new faction or alien species with a different ideology and goals, they decided to reskin or do a spin off of things already established in the lore. The covenant storm, was really just an excuse to have the same covenant species in the games as enemies to shoot at. They thought "the players just want to shoot at elites and grunts just like before fuck yeah." They didn't have enough faith in themselves to bring in something new to replace them. So to sort of satisfy the itch of a new threat, they brought in the prometheans, which to me, were always boring and overhyped reskinned forerunner sentinels. They had no culture, no society, no significant history other than being former humans that were a part of one of the didact's experiments against the flood. The Ur-didact being a villain was more of the same thing. They didn't have a central villain anymore since truth and the gravemind were gone, so rather than come up with new central villain; perhaps the leader of a new alien race/faction, they decided to do a plot twist of something that was already established in the bungie lore: the didact. As with the banished, same thing as the covenant storm. Rather than come up with something new, they simply brought the same old covenant species back under a new banner to try to spice things up a little. They said things like "theyr'e not covenant". Like ohh yea these guys are definitely not covenant right? They don't believe in the great journey so they're not the same covenant races we've been fighting for nearly 20 yrs. Right guys? Because at the end of the day, they don't have enough faith in themselves to create new entities both in the lore and in the games to match or even beat the bungie-created covenant races. Lastly, cortana being the robot overlord of the galaxy. . . This is again, just a result of them playing it too safe. They didn't know what to do. They couldn't come up with anything new, so they took a very 80's Sci-fi approach and did a very poor spin off of a character already established in the lore: Cortana. This time however, it was a screw up several magnitudes worse than the previous ones. I just don't think they believe in themselves. They don't think that they can ever surpass bungie era Halo. Which is why I think the games did not take off, at least from a lore perspective.
AND DEY SAY ZULUL AND DEY SAY ZULUL AND DEY SAY ZULUL