Lawrence H. Miller was born 26 April 1944 to Mary Lorille Horne and Howard Hanley West. His parents divorced in 1946, and in June 1948 his mother married Frank Soren Miller, who legally adopted Larry in September 1949. Larry attended Salt Lake City public schools, graduating from West High School in 1962. He married his high school sweetheart, Karen Gail Saxon, on 25 March 1965, and they have five children: Gregory Scott, Roger Lawrence, Stephen Frank, Karen Rebecca, and Bryan Joseph.

Following graduation from high school, Miller was employed in construction by his uncle William Reid Horne until 1964 when he went to work for American Auto Parts. Two interests, softball and drag racing, helped launch his careers in sports and automobiles. From 1963 to 1970 Miller raced cars, and from 1962 to 1985 he was an outstanding fast-pitch softball player, pitching in the Salt Lake City Metro League and the Denver Metro League.

In 1966 he became parts manager and later manager of the parts and service departments for a Utah auto dealer. In 1970 he moved to Denver to play softball and work as a parts manager for two Denver Toyota dealerships. In 1978 he was promoted to operations manager over five Toyota stores.