A new, dramatic trailer for the upcoming premiere season of the U.S. X Factor shows Simon Cowell getting in a fight with a fan ... a war of words, that is.

Still, it clearly gets to Cowell -- after he can be seen telling producers that the show is "not working" and has "gotta go."

Of course, we're sure Simon wouldn't just walk away from his million-dollar baby.

Check out the trailer above -- will you watch the Sept. 21 premiere?