The Hubble Space Telescope NICMOS Camera 2 was used for H-band imaging of 12 BL Lacertae objects taken from the larger sample observed with the WFPC2 in the R band by Urry and coworkers and Scarpa and coworkers. Ten of the 12 BL Lacs are clearly resolved, and the detected host galaxies are large, bright ellipticals with average absolute magnitude =-26.2+/-0.45 mag and effective radius =10+/-5 kpc. The rest-frame integrated color of the host galaxies is on average =2.3+/-0.3, consistent with the value for both radio galaxies and normal, nonactive elliptical galaxies and indicating that the dominant stellar population is old. The host galaxies tend to be bluer in their outer regions than in their cores, with average color gradient Δ(R-H)/Δlogr=-0.2 mag, again consistent with results for normal nonactive elliptical galaxies. The infrared Kormendy relation, derived for the first time for BL Lac host galaxies, is μe=3.8logre+14.8, fully in agreement with the relation for normal ellipticals. The close similarity between BL Lac host galaxies and normal ellipticals suggests that the active nucleus has surprisingly little effect on the host galaxy. This supports a picture in which all elliptical galaxies harbor black holes that can be actively accreting for some fraction of their lifetime.