david weaver
If it's not just the surface, I'd bet silica. I've had the same issue with salvage redwood from my porch (this was cladded framing lumber for sills, etc, and wasn't exposed to elements or airborne dust).

If it's just the surface, I'd bet dirt.

If wood has silica in it in particles large enough to make visible nicks, you can usually see the particles under light.

Getting the heavy work done without too much tearout and then using the cap iron will save you some time (you can take bigger bites and keep a dulling iron in the cut further into dullness if the cap is set right), but you can't get around the damage. Some HSS irons tolerate it a little bit better, but they'll still dull.