Gotta go with Skeletons here, creepers are annoying yes, but easy to deal with unless the sneak up on you.
Skeletons however can hit you from afar which can make it harder for you to get to them, plus, they have killer aim so you pretty much get shot every time unless you duck behind some blocks.
