Java performance has always been one of the primary concerns from the time Java became available. Often times it meant having to use faster more well implemented heuristics to reduce the memory/CPU requirements.
The cases where programming in C++/C for performance reasons have been greatly reduced over time, but there are strong cases to be made when specialized libraries needed are not available.
The compute speed of Java is more than sufficient on newer hardware, but the thing that can adversely affect performance is memory usage and the garbage collector. Here is an excellent article on the current garbage collector:
This was really a pretty extreme case, but from experience the garbage collector also needs CPU cycles and room to run, and if it has to span a multi-gigabyte heap it will exhaust itself before accomplishing as much work.
Keeping the memory footprint smaller and making it easier for the collector to free-up memory can make this a non-issue. Applications with a lower unused heap will trigger collection sooner and it will not force an all-stop. Increasing the ceiling will potentially delay collection, but it will be less efficient when it runs and can actually make performance worse. Pegging the heap and CPU is not a place you want to be.
Entraspan has developed several servers that can run in a 40-60 MB heap for months at a time. Even our largest application servers will only be given a few hundred megabytes. Memory usage will fluctuate but typically less than 200 MB will be in use at any given time.
The key is to keeping the minimum amount of data in memory by using proper caching and aging out data that is no-longer in use and keeping the active data to minimize database load.