Yarborough was born in the small town of Timmonsville, South Carolina, in 1939. His father was killed in an airplane crash when Cale was just 12-years-old, and as the oldest of three sons, he was forced to grow up fast.
He spent some time as a semi-pro football player and as a Golden Gloves boxer, but it was auto racing where he would make a name for himself. After attending the second-ever Southern 500 at Darlington Raceway (without a ticket) in 1951, he would later attempt to make his NASCAR Cup debut in one of the sport's most physically-demanding races. However, he was still a teenager and lied about his age. The sanctioning body found it, and he was disqualified. Now there's a garage at Darlington named after him...