Walt Brown was a graduate of Bemus Point School in 1933 where he lettered in basketball, cross country, track and baseball. He played semi-pro baseball around the Jamestown area with Sheppard Furniture and the Steel Partition Bombers before breaking into professional baseball with Ashville, North Carolina of the Southern League in 1935. He then went to Albany, Georgia of the Georgia-Florida League and later to Huntington, West Virginia, before signing with the St. Louis Cardinals.
With the Cardinals' Bentonville, Arkansas team of Arkansas State League, he won 13 and lost 3 which earned him an invitation to the Cardinals' spring training camp. He eventually pitched for Daytona Beach, Salina, Kansas, and Kilgore, Texas where he was married at home plate in 1939.