Context...doesn't matter

Look, if you’ve been successful, you didn’t get there on your own. You didn’t get there on your own. I’m always struck by people who think, well, it must be because I was just so smart. There are a lot of smart people out there. It must be because I worked harder than everybody else. Let me tell you something—there are a whole bunch of hardworking people out there. If you were successful, somebody along the line gave you some help. There was a great teacher somewhere in your life. Somebody helped to create this unbelievable American system that we have that allowed you to thrive. Somebody invested in roads and bridges. If you’ve got a business—you didn’t build that.

Here's why the context of the "you didn't build that" comment doesn't help Pres. Obama.
If the "that" he was referring to was the business you've got -- a grammatically sound interpretation of the remark -- it shows, at best, economic ignorance and, at worst, hostility towards free market capitalism.

But let's give him the benefit of the doubt and concede that it was just a grammatical error; let's accept that he was referring to roads, infrastructure, the American system, etc. Under that interpretation, it shows that Pres. Obama believes that we the people don't own an investment in our own government. We didn't lay the asphalt, fire the steel, or teach the leaders of tomorrow (even though some of us actually DID do that), so we didn't build it. Government put it all in motion (except for those handful that actually DID those things); we just paid for it. 

When was the last time Pres. Obama, or any member of Congress, pulled out a wallet and said, "This school construction project? Yeah, that's on me," and then picked up a hardhat and went to work? The government doesn't do these things unless they can stick us with the bill. So, while maybe we didn't physically build it (except, again, for those people who had a direct hand in it), our financial contribution to it is reduced to nothing, and government gets all the credit.

Let's finally put all this equivocating aside. People have a right to be incensed with the president for saying, "you didn't build that," because no matter how you slice it, it's damned insulting.