In a horizontal cluster, JVMs that reside on different physical machine are clustered. The obvious advantage to a horizontal cluster is that if one physical server goes down, the other physical server can continue to produce the applications in the JVM, resulting in high-availability. Also, from a systems engineer perspective, a horizontal cluster lets the engineer do work on one cluster member without bringing down the applications being produced by the JVMs in the cluster. For example, the engineer could bring down the server01 JVM, patch the server, restart the JVM, and bring it back up, without causing an outage, as the server02 JVM can still produce the application.
In a vertical cluster, JVMs that reside on the same physical machine are clustered. Similar to a horizontal cluster, a vertical cluster lets the engineer do work on one cluster member without bringing down every JVM. However, if the physical server goes down, the entire cluster also goes down.