Adam Scott guarded his children from his fame when they were little.