That overhang is most likely not plywood.
Personally I would frame them as you described, but with pressure treated material, then cover the framing with pressure treated 1"x8" material for the risers, because they'll be contacting concrete, and could wick up moisture from it.
It looks to me as though the builder cut his plywood a few inches short for the tread material, and probably used some solid pine to face them with. I would secure the pine to the plywood with biscuits and glue.