Door Panel Material Question

[ACovelle]

I was detailing the inside of my 2019 limited today and went to clean off the drivers side armrest and handle, which looked as though it had accumulated a ton of grime after a cross-country drive. I rubbed in some LEXOL all-Leather cleaner to the handle and was disgusted at how black the towel came out. So i kept cleaning it...and it kept coming out nearly black and the leather looking grey and faded. Thought it was just deep grime at first but now im thinking i rubbed off some dye in the leather and am hesitant to use this product or any chemical cleaner on anything other than the vented seats.

Anyone have experience with this kind of thing, or seen this happen? Are those door handles some kind of dyed synthetic leather and i just took all the dye out?

Thinking now i should be sticking to just soap/warm water

thanks