What do you think is the most terrifying thing to happen so far in the game?
You get the walkers for sure but it can also be kinda scary when there is a hostile group around like the bandits or Carver chasing you, I also found going into Crawford a daunting prospect before you find out it is overrun. Plus there is also the occasional psychopath like the 'Guy on the radio' who was scary at first but then he just turned out to be some sad dude, oh and don't forget Nate too! He's a hell of a nutjob.
For me I think the scariest is the massive zombie hoard that follows Lee's group to Savanah via the railroad track, there was thousands of them and it's the same hoard that Lee has to fight through in the last episode of the season. You first see them when Ben shouts "There's something coming!" and it's what looks like a massive swarm of bees on the horizon!