• Play the game fairly.
Getting reported for cheating/hacking can lower your Trust Factor.

• Get commended.
You can retain a Green Trust Factor by getting commended. Remind other players to give commends if you feel like you deserved it.

• Don't be toxic.
CS:GO is taking steps against toxic players and besides degrading your Trust Factor, you could even receive a game ban from it even from Casual.

• Don't get kicked.
If you're getting kicked for griefing or other obvious reasons, great!
Otherwise, asking your teammates to kick you just so you could abandon a game without receiving a cooldown will also degrade your Trust Factor.

• Do some Overwatch.
I can't confirm this, but I've heard that legitimately solving some Overwatch cases will bring back your confidence factor. Feel free to try it out once you've unlocked Overwatch cases.

• Play non-competitive games.
Try playing Casual, Deathmatch or other casual gamemodes until your Trust Factor goes Green again. Your Trust Factor can still be affected in casual gamemodes.