There is a "culture of discouragement" in the black community. This culture is handed down from father to son, mother to daughter. Underlying this culture of discouragement is one of the worst crimes any human being can commit against another, short of murder or rape, and that is to deny a person's humanity and to do so on a society wide basis with laws used as a weapon. This is, in fact, what has happened to black people in America. The culture of discouragement holds that the deck is so thoroughly stacked against black people that there is no reason to try. Each older generation, having tried and failed in various ways, has a vested interest in passing along this culture to those who are younger, so that the cycle perpetuates itself again and again.
There is also a pact of social cohesion within the black community not to aspire to "being white", not to try to be like white people. If a black student does well and intends to attend an elite college, he will likely be cut off from peers. He will be taunted as wanting to be white rather than sharing the sense of oppression felt by everyone else. Who do you think you are? You think you're better than me? To aspire mentally and academically is to risk being different in a way that is not acceptable.
This kind of social pact exists within many other subgroups as well. Working class white kids are not expected to go to Princeton and they, too, can be ostracized for having such aspirations. Their teachers, having not attended an elite school themselves (jealousy) can attempt to discourage the student and undercut their belief in themselves with a variety of cutting remarks, many uttered at some venerable point in the students life when he might be feeling less confident.
These facts just scratch the surface.
We have a system of higher education that is designed of, by and for the wealthy classes. This system perpetuates advantage, which is exactly what it is supposed to do. The strongest upward ladder for the poor is to have a family member who is successful and can pave the way to job opportunities. Without that, experience and resumes are just words on paper. The door must first be opened for someone to walk in and blacks consistently find that the door is closed and that they lack the means to open it on their own.