"You're either with us, or against us" is sometimes an acceptable rhetorical excess. But it is never a truthful statement. It is never a rational thing on which to insist.
For instance, George W. Bush used it just after 9/11, in the context of the terrorist Jihadist atrocity. Everyone who heard him knew what he meant. Still, human memory is such that today many Collectivists roundly criticize Booooosh for this statement, pointing to it as part of his ridiculous cowboy swagger and simplistic thinking. They forget, we suppose, that Senator Hillary Clinton used the same formula several days before Mr. Bush.
The problem is that "You're either with us, or against us" ignores reality, and it ignores human nature. Which, I assert, is dumb for us to do.
Human beings are not binary creatures. We are pretty much NEVER EITHER "on or off" on anything. We may be VERY strongly one thing or another, but we are always a hodge-podge of conflicting feelings, thoughts, etc. On a host of things, we may be rather indifferent.
Virtually always, people on any given question will fall out somewhere on a bell curve. It was true in the American Revolution with respect to the question, "Shall we remain loyal, or shall we form a new nation?" Some were for loyalty to the Crown, some were Revolutionaries, and the vast middle was some degree of indifferent. Even a lot of the "kinda Revolutionary" people were conflicted. Pick your historical question; in pretty much every situation, you'll be able to apply the bell curve to human beings being human.
Just now, there seems to be a lot of very loud insistence that we ignore the bell curve of humanity, which is dumb. With respect to Islam, I am hearing and reading a lot of stuff insisting that Muslims prove to us that they are "moderate" (or do not support Islamism), down on the individual level. They have to go out in the street, essentially, and make it their life's work. It isn't a wrong thing to want; it's just silly. It isn't remotely going to happen under any circumstance because of the bell curve.
For the vast middle of the bell curve, Muslims world-wide are not with us or against us. They essentially don't care enough either way. Even many of those that might harbor some animosity toward us just don't feel strongly enough to do us any harm. Here is where we have an opportunity...good or bad. We MUST appreciate that we are in a battle MOSTLY for the bell curve. There are stupid, wrong things we can do to shift the vast middle against us, and there are intelligent, right things we can do to shift the middle towards us. A little shift means MILLIONS of potential enemies or "not enemies".
The same holds true with a lot of rhetoric I am hearing in the political fray. Even among people who see themselves as conservatives, there's a lot of "With us or..." nonsense being thrown around. We have to bear in mind, those of us that are "true believers", this is ALWAYS about PR...ALWAYS about a message that can be successful at moving the vast middle of the bell curve toward us. That does not mean...EVER...we surrender anything; it means we find ways to show people who we are, what we believe, and that those a good enough to move them out of indifference toward us. It also means we don't alienate our friends near us on the bell curve with excessive rhetoric.
Honoring human nature is always a wise course. Remembering that humans are NOT binary things, but things that fall out in almost any case on a bell curve, with the vast majority of the population being indifferent...that is wise. It gives you a "reality tool".