Hate having to be a common racketeer as part of your Theives Guild hazing? Well, this isn't going to let you circumvent that to so you can feel guiltless, but it does let you kind of heal the scars by performing certain quests.

Keerava and Talen-Jei will forgive you if you help them get together (Complete "Sealing the Deal")

[UNTESTED] Bersi Honey-Hand will forgive you if you help the citizens of The Rift (Be on Stage 15 of "I Done Got Thaned!")
While realistically I imagine beating him up and then becoming a politician that governs him would be more reason to dislike you, you do get in that seat by HELPING the hold's peoples...s'pose you can go back to his shop and buy a lot of his wares to make up the monetary deficit, too.

As for Haelga...she simply won't forgive you. You're an outsider drifting into Riften with the gall to think you can stay at her working man's inn, and then you join up with the lowlifes in the Ratway and ruin her day? She's not going to get over that!

While you can do these quests before or after joining the Thieves Guild, I would suggest that you make up for the crimes AFTERWARDS.
There's no penalty for it, but I imagine that immersion-wise, they would have more reason to dislike you if you betrayed them!

Incompatible with other things that edit these quests or dialog topics.