Tavis Smiley says he was ousted as the speaker at a Martin Luther King luncheon because he was trying to hold the president accountable.