Well, I did too, and the only actual reason I went down is because the game does not automatically grant short rest benefits when long rest is interrupted but you spent more than one hour, as with the actual tabletop rules.
So my party was also level 3 but out of everything barring a Fog Cloud on my Ranger (Hunter Ranger, Champion Fighter, Survival Monk, Tree Warlock), not even full life... And I did manage to survive 10 rounds.
That encounter is a dual lesson: always take a short rest before starting travelling if you're missing resources / HP (at least until game fixes aforementioned problem) AND *think tactical*.
Unless you happened to be ambushed in the "plain plains map" in which case yeah, indeed you're screwed.
As long as you have *any* kind of elevation point requiring climbing, you can win this. Dodge/Dash your way ASAP until everyone is up and blocking access points, then focus fire with highest AC people while Doding/healing with those who are critical. It's costly, and you'll end up completely empty of everything possibly, but it's doable. ^^
If you're lucky to have a vantage point which is not opened on all sides even easier although I didn't meet any such map in random encounters to be honest. Same if you have any AOE spells (Fog Cloud to force them to fire from one direction or make them waste a full round moving away, Spike Growth of course to kill a few, Entangle to stop a handful, or just plain old Shatter to weaken them).
Monks also shine here, if you have a few Ki points you can aggro several of them to split them, then Disengage and Dash to backtrack when you are on the verge of getting surrounded. Otherwise, putting it as the easiest target for ranged attacks and Dodging ensures he can sustain a LOT of attacks with nearly 0 damage (Deflect Missile is great).