You won't regret it - the Sierra in California is stunning. You'll be close (couple of hours) to Yosemite which is breathtaking but also incredibly packed with people.
There's a direct highway from the LA Basin straight to an exit in the Sierra that's only about 5 miles from Mammoth. From memory it takes about 5 hours to get there, but the change in scenery from LA to the Sierra is unforgettable. A YouTube video is easily found online for the entire drive.