I never got into a confrontation like that. On public land I'm usually the first one to get up and go looking for a new spot when there's an encroachment. I've hunted Dan'l Boone (Beaver Creek) and all through the BSF.
I've had nothing but nice encounters on public land when it's been about talking. The problem is that the majority of encounters I've had on public lands while turkey hunting. . . Well, I didn't know anyone was there until the shot. Yikes! If this happens to you, make sure you settle down before talking to the guy, or maybe just leave without saying anything. You probably need to change your shorts anyway, and whatever was going to be said wasn't going to be charitable either way.
My advice to anyone on public land is this: Everybody is armed. Everybody needs to be polite. If the other guy isn't, it is all the more reason for you to be. Back off and go somewhere else. Show your palms. Keep your eye level below theirs. Be calm and be calming. If the other party gets too randy, call 911 as soon as you're safe and have them arrested for assault, menacing, or whatever they did. Telling you to leave is one thing. When they get into your space, undo their safeties or show you their gun butts, they need to go have a time out at the state's expense.
On the other hand, I'm quite confrontational when it comes to my own property. It's posted. Folks know it's posted. If you come over the fence, you and I both know you meant to do it. If you say your lost, I'll let you run back over the line, but it had better be a straight line, you'd better be quick about it and you better not get lost again. The county dispatcher and the CO both know me on a first name basis, and the sign out front says it all:
"Trespassers will be tormented to the fullest enjoyment of the landowner before being prosecuted to the fullest extent of the law."
On private land: If you're a trespasser and you get caught, you get whatever is coming to you. Be extremely nice and try your best to negotiate your way out without the authorities being called. You both know what you did; do not insult his intelligence. The landowner hold all the cards. Do NOT, hassle the landowner on his own property. Treat him as you would a vengeful god. Otherwise you could end up in a shallow grave. I've gotten hassled on my own land a couple of times. In one case, I told the guys they had 10 minutes to make up their mind what they were going to do, but I was going up and locking the gate and calling the sheriff. They were gone in under 5. In another I went over to offer some friendly advice to a lost soul, and I had a gun barrel pointed at me. I got the guy into a conversation until he lost focus and turned the barrel away. When he looked back, he was staring at my gun barrel. I'm not sure what that guy was thinking, but he sure knew what I was thinking when he looked back. I had a good ten minutes with that barrel pointing at me; I repent none of my response. My only words when he looked back were "Do you know your way outta here or do you need help?" He didn't need help.
If you're on private land by permission and it ain't yours, treat it as public, but you first call should be to the landowner if there is trouble. If you're hunting my property, I want to know about it first. I'll work something out. I know the sheriff, an ADA, the CO, the dispatcher -- and the coroner. Be extremely careful if you're not family and you tussle with the kin of the landowner. You're probably going to lose, and you probably don't want to hunt there anyway.