May I scrub my face with an exfoliating towel?

Now that winter has arrived, my skin has become rough, so I scrubbed it with an exfoliating towel. Afterward, my face started to glow and my skin looked great. I remember actor Yoon Da-hoon appearing on the TV show "Let's Live Together with Park Won-sook." When the female cast members asked him how he maintains his glowing, healthy skin, he replied that he scrubs his face with an exfoliating towel occasionally, which is how his skin stays radiant and healthy. Since facial skin is delicate, wouldn't scrubbing it with an exfoliating towel damage it? I'm asking because I also feel like my skin becomes radiant and better when I scrub it once in a while.

Still, I was worried, so I didn't keep pushing it.

To be honest, since dead skin cells started flaking off my face like dirt as soon as I used the exfoliator, I really wondered what was in it causing this. I would like to ask if scrubbing my face with a washcloth damages my skin or if it is okay to keep doing it.