The IOTV I wear when I go outside the wire has soft kevlar panels that will stop 9mm by themselves, in addition to the plates. If your plates get hit or you dive onto something hard in a gunfight, they can crack. If they crack, they won't function properly. What stopping power they still have left may slow the bullet down enough for the kevlar to stop it.
The problem with adding layers is that it adds weight, heat and reduces maneuverability. It also increases your likelihood of you being noticed for wearing armor. You have to balance protection with everything else.