We analyse the orbital distribution of elliptical (E) and lenticular (S0) galaxies using SAURON integral-field stellar kinematics within about one effective (half-light) radius. We construct the anisotropy diagram, which relates the ratio of the ordered and random motion in a galaxy (V/σ) to its observed ellipticity (ɛ), for the 48 E/S0 galaxies from the SAURON survey. For a subsample of 24 galaxies consistent with axisymmetry, we use three-integral axisymmetric Schwarzschild dynamical models to recover the detailed orbital distribution, and we find good agreement with the anisotropy derived from the (V/σ, ɛ) diagram. In a companion paper (Paper IX), we show that the early-type galaxies can be subdivided into two classes of systems with or without a significant amount of specific stellar angular momentum. Here, we show that the two classes have different distributions on the (V/σ, ɛ) diagram. The slow rotators are more common among the most massive systems and are generally classified as E from photometry alone. Those in our sample tend to be fairly round (ɛ <~ 0.3), but can have significant kinematical misalignments, indicating that as a class they are moderately triaxial, and span a range of anisotropies (δ <~ 0.3). The fast rotators are generally fainter and are classified as either E or S0. They can appear quite flattened (ɛ <~ 0.7), do not show significant kinematical misalignments (unless barred or interacting), indicating they are nearly axisymmetric and span an even larger range of anisotropies (δ <~ 0.5). These results are confirmed when we extend our analysis to 18 additional E/S0 galaxies observed with SAURON. The dynamical models indicate that the anisotropy inferred from the (V/σ, ɛ) diagram is due to a flattening of the velocity ellipsoid in the meridional plane (σR > σz), which we quantify with the β anisotropy parameter. We find a trend of increasing β for intrinsically flatter galaxies. A number of the fast rotators show evidence for containing a flattened, kinematically distinct component, which in some cases counter-rotates relative to the main galaxy body. These components are generally more metal rich than the galaxy body. All these results support the idea that fast rotators are nearly oblate and contain disc-like components. The role of gas must have been important for their formation. The slow rotators are weakly triaxial. Current collisionless merger models seem unable to explain their detailed observed properties.