I have this sense as well, that something was lacking from this film at least in comparison to Iñárritu's previous little opuses. Maybe there is nothing lacking. Maybe these movie is so true to its attempt to convey the disconnect between cultures, language, and lack of language that it cannot be completely understood until we can empathize to some degree with the different cultural and linguistic contexts the characters are in.
Maybe until we've herded sheep in Morocco (or wherever they were) or experienced deafness or been an illegal alien we can't really begin to understand Babel.
But is that what Iñárritu intended? Part of me thinks he wanted to portray characters that we all could relate to in some way, despite their and our differences. I don't think he pulled this off though.
So maybe I am just making excuses for Iñárritu because I don't want to admit that he may have faltered in some way on Babel. Maybe too much money and big stars was not a good thing.
Maybe.