What people think about you, what they are saying, what they are doing. God allowed it so He can show his greatness through you. When they think you are finished, he’ll come rescue you.
Romans 8:28 And we know that all things work together for good to those who love God, to those who are the called according to His purpose.
There might be pain now but joy comes in the morning. Don’t wish them away, somebody must witness your success.