I don't think the problem is just that of Battlefield. Like She Carnage said, I believe the problem expands to all games with game modes that are objective focused. Alot of times the people playing in them are just doing their own thing, having fun or just learning or experiencing the new maps/vehicles the game or the game itself. Relying or randoms to complete goals & objectives can trully get frustrating more so the higher the number of players in that game. Its just about always better to game with a group of people you know than by yourself, especially for obective/team-based games. Probably one of the areas that makes CoD more popular than Battelfield, less need for good team members. Its also the only reason why the only Battlefield game I own is 1943, no friends into it.
As for the main cause of playing with unskilled or stupid people I imagine the core of the problem is that most shooters don't have a "proper" ranking skill system. The lack of such a system allows skilled people to play with and against unskilled people. The advantage of that is it allows for matches to start significantly quicker, especially for the older games. The cost of is that you are significantly less likely to game with only high quality/skilled gamers.
However there are games like Halo that actually do have proper ranking system "True Skill" that allow only equally skilled players to game with eachother, (once your reach a high skill). However that system too can be bipassed if a skilled player is partied up with an unskilled player forcing the system to find a closely-leveled unskilled opponent to balance out the unskilled party member of the skilled player.
If there was a mode that allowed people to be seperated by either Stupid Fun or Hardcore Competion the gaming community in those games would definitely be better off with it.