If we are comparing chimps to humans, a more reasonable comparison is with primitive man, not modern man.
Clearly there are differences (language and abstract thought seem the most important to me), but modern science is doing a lot to understand exactly how we are different, and how those differences evolves.
Here is a fascinating paper on just that:
The human HARE5 and the chimpanzee HARE5 sequences differ by only 16 letters in their genetic code. Yet, in mouse embryos the researchers found that the human enhancer was active earlier in development and more active in general than the chimpanzee enhancer.Proof here that a small change in the DNA has a dramatic difference in the brain, given more evidence that we are closely related to chimps.
All told, human HARE5 mice had brains 12% larger in area compared with chimpanzee HARE5 mice. The neocortex, involved in higher-level function such as language and reasoning, was the region of the brain affected.