While off roading, or at least trail riding, around here I'm glad the cab and box have a great deal to move without dammaging each other. If these two main components were close together the movement of them would cause damage even on some easy trails.
All pickups have some articulation between the cab and the bed. It's hard to make a frame that's that solid that doesn't add a lot more weight. The rubber strip's primary purpose, according to the manufacturers, is to make the gap more aerodynamic. As air flows past the side (and the bottom) of the truck, it tends to blast in through the space in front of the bed, and out the top behind the cab. This, combined with the turbulence generated, creates drag. Putting the rubber "dam" or "seal" inside there (I hesitate to say seal because it's not really a tight seal) causes that air flow to slow down, almost stop. Damming that gap up doesn't completely get rid of the turbulence or the air flow behind the cab, but it cuts the vast majority of it out and reduces drag a little bit.
I suspect the reason that the rubber doesn't quite touch the cab is because if it did, it would drag fine grit across the cab's paint job. So leaving a small gap makes a lot of sense.
I cannot claim that this is exactly the reason it's this way, all I have is my memories of discussions from dealers and literature in the past.
Air suspension autoleveling... I did notice today the gap was more, and after I thought about it alittle I was happy because of the amount of difference the front and back can have because of air suspension
My 2013 F150 did not have this, but my 2014 Ram Sport had the same thing and my brother's 2013 Ram Laramie also has it....now my 2016 Rebel has it as well. What you should be more concerned about it the "weatherstrip" on the bottom, that runs across the frame (right to left) between the cab and box. Most people do not realize it is there and it rubs the paint off from the cab. I can almost guarantee that as you're reading this, the paint has already been rubbed off. Do a video on that one.