Born to German immigrants Johannah and Rich, Carle's mother grew homesick for Germany and moved the family back to Stuttgart when he was six. He was educated at the local art school. His father enjoyed drawing and would often share sketches with his son.
His father was drafted into the German Army to fight in World War II. He was captured by the Soviets and returned a “sick man," Carle said, “psychologically, physically devastated" and weighing only 85 pounds.
One of Carle's art teachers opened his life to the “Forbidden Art" of German Expressionists and Abstract Artists. This action could have landed Herr Krauss in jail. The government considered these people to be dangerous degenerates.
At 15, Carle was conscripted to dig on the Siegfried line, a network of trenches and bunkers stretching 390 miles along the western border of Germany. He saw combat victims up close. His home in Stuttgart was left without a roof, doors or windows, but it was the only one remaining with walls.
In 1952, he arrived in New York City with $40 in his pocket. He got a job as a graphic designer for the New York Times. He was drafted into the US Army during the Korean War and was stationed in Germany as a mail clerk.
He returned to New York and his old job at the Times. Eventually he became art director at an advertising agency.