I'm going way out on a limb here, and claim both.
There is an interplay between the two. Wood, for example in a dark and dry place, can last for a very long time, but I doubt there are many wooden structures that are, say 2000 years old. Perhaps some supports for tunnels in dry climates. Those organic compounds will break down over time. The method of using the wood, how it's protected from its environment is important.
Stone is another obvious example. Plenty of stone buildings have been built and crumbled, but the really well cut, fit, and stacked stones seem to last a long time. There are countless temples and castles that are simple stone with very tight tolerances (methods) that last a long time. The pyramids are a spectacular example.
Hoover Dam is reinforced concrete, and is expected to last a long time. I suspect the steel will rust away, but the compression from the water will keep the concrete stable. (not a civil engineer, but I'd bet this was seriously thought about by civil engineers when it was built)
The materials are important, but they can be misused, and master craftsmen can use them far better than I ever will, So the methods matter as well.
_edit_
I looked it up, hoover dam used steel pipes, not solid bars, so there's room for the corrosion to expand into the void created by the pipes.
Master craftsmen I tell ya, they think hard about that kind of stuff.