Or if it's made of stone. Stacking giant stones on top of each other is a sure-fire way to make a building outlive you.
After that, the longest-lived buildings that I am aware of are made of wood. The catch is they've been rebuilt 50 times, once per time they burned to the ground.
After those, the longest-lived buildings are made of Roman concrete that we can't reproduce. (To give you an idea how insane Roman concrete was, you can go kayaking north of Naples, and kayak through a concrete Roman building that is sitting on piles in the Mediterranean sea)
The materials are important, but they can be misused, and master craftsmen can use them far better than I ever will, So the methods matter as well.
_edit_
I looked it up, hoover dam used steel pipes, not solid bars, so there's room for the corrosion to expand into the void created by the pipes.
Master craftsmen I tell ya, they think hard about that kind of stuff.
That claim seems to date to a particular article written in 2017 that wasn't well sourced. Roman concrete is interesting stuff and has useful properties, but humans have since created concrete mixtures that are far superior. But they're expensive, so it's not too surprising we don't see them getting used in buildings that compare less than favorably to a temple built a couple thousand years ago. Survivorship bias taken to the extreme.
Modern concretes can do a whole lot of stuff Roman concrete can't, because there are so many formulations of it. But if you want to stick a building literally in the ocean and have it never ever disappear, nobody has shown that we can actually do it today.
There's a whole lot of theory and talk by experts, about how we don't need to make it, but if we wanted to, boy would it be easy, but don't worry, modern concrete is just so amazing, you should just use that, for modern use cases, and oh by the way, it would be too expensive to make, even though we haven't actually made it or tried to bring the price down.
There's a world of practical experience needed to claim for a fact that modern concrete is legitimately better, much less that we can actually make it and that it would hold up as we expect. I'm still waiting for concrete evidence.
The bigger difference in components is the kind of cement the Romans (and we "moderns" until a few years ago) used, i.e. pozzolanic cement, nowadays everything is "portland" cement.
BUT the definite difference is the kind of structures, Romans did not use "reinforced" concrete, only various types of "plain, non-reinforced" concrete, and all their structures are based on the main characteristic of concrete, which is its resistance to compression.
The idea of reinforced concrete is all about adding to a material with excellent compression resistance (but no resistance on tension/traction) a material (steel) with excellent resistance to tension/traction and relatively poor (in the quantities used in reinforced concrete) resistance to compression, obtainining a composite material that excels in both.
About ashes, overall it is more about their size that about their nature, concrete is a composite and if you have all possible sizes of aggregates (ashes are very, very small sized particles) in the "right" amount you essentially fit "better" the space, i.e. you have a higher density of the resulting composite, and, particularly when compression resistance is the goal, the higher the density the better the resistance.
Imagine (say) that you have to fill a 100x100x100 mm box with 10 mm balls, you can fit in them a certain amount of these balls (roughly 10x10x10=1000), but you are leaving lots of "air" between them, a single 10 mm ball is 2/3x3.1416x5^3=262 mm3, so the 1000 balls total 262,000, but the volume of the box is 100x100x100= 1,000,000, now if you have some 2 mm balls you can add them in the same volume, and then if you have some 0.5 mm balls you can put some of them in that same box as well, etc.