Ginkaku-ji [lit. "Temple of the Silver Pavilion"] is just a nickname, the official name of this place is Jisho-ji - "Temple of Shining Mercy". The pavilion was originally built for shogun Yoshimasa to relax in solitude and read books; the library room is oriented is such a way that the reflected moonlight from the pond in front of the building suffices for reading at night. Yoshimasa wanted to cover the facade in silver foil to emulate his grandfather's Golden Pavilion, but the plan got delayed and the Silver Pavilion stays "unfinished" for the past 530 years and therefore unwillingly adheres to "wabi-sabi" ideology.
This gravel mountain is a part of Ginkaku-ji garden and it's called Kogetsudai [translates as "moon-viewing platform", maybe because the shape resembles the moon's reflection on water]. Some people say it represents Fuji mountain, but I wouldn't draw conclusions too fast: Zen buddhist symbolism is usually more complex than that.