La specifica è molto chiara a riguardo
2.5.3. Heap
The Java Virtual Machine has a heap that is shared among all Java Virtual Machine threads. The heap is the run-time data area from which memory for all class instances and arrays is allocated.
The heap is created on virtual machine start-up. Heap storage for objects is reclaimed by an automatic storage management system (known as a garbage collector); objects are never explicitly deallocated. The Java Virtual Machine assumes no particular type of automatic storage management system, and the storage management technique may be chosen according to the implementor's system requirements. The heap may be of a fixed size or may be expanded as required by the computation and may be contracted if a larger heap becomes unnecessary. The memory for the heap does not need to be contiguous.
A Java Virtual Machine implementation may provide the programmer or the user control over the initial size of the heap, as well as, if the heap can be dynamically expanded or contracted, control over the maximum and minimum heap size.
The following exceptional condition is associated with the heap:
If a computation requires more heap than can be made available by the automatic storage management system, the Java Virtual Machine throws an OutOfMemoryError.
Per rispondere alla tua domanda:
When we say "the heap" in the context of the JVM, are we talking about somewhere inside the VM itself (which may or may not be allocated on the heap of the underlying physical computer)? Or do we refer to the heap of the physical computer?
Quando parliamo di heap, parliamo di memoria già allocata per la JVM. Quindi, è dentro (?) La JVM (se questo ha senso dire). La JVM ha la sua gestione della memoria.