Here are some of my tips on preventing hackers and griefers.
1. Plugins/How to tell if they are hacking- Plugins will stop some hacking. dev.bukkit.org/server-mods/nocheat/ <--- That's NoCheat. It should help with hackers. Remember some hacked clients have NoCheat detect so that will not stop all the hackers. Another way is to get some people to stop hacking is to know what hacking looks like. I am not sure how to say it but like if you have a plugin that tells you when they find diamonds that will help stop X-ray. If you see they get a lot of diamonds in 0% visibility then they are using X-ray. And remember to get a jail plugin because if you just put someone in a bedrock block they can hack out of it. Another way is to get a hacked client and mess around with them on singleplayer. Then you will kinda know what it looks like while hacking and stuff. Also get a plugin that prevents creeper explosions, flint and steel, TNT, lava placing, and all the other griefing tools.

2. Griefing- If someone griefs they will most likely take everything important or not. Rolling back the area that got griefed might help. World Guard might help a lot. Getting plots or factions might help so griefers cant destroy homes.

3. Admins/mods- Admins and mods will help A LOT. Find people who can stay on for a long time and make sure you can trust them. Having mods and admins that live in another country will help. Then when you are asleep they will be awake watching the server.

4. Fake?- When someone says they are from Planetminecraft and they want to review your server or test the commands its a fake. They just want admin so they can grief the server. Make signs at spawn saying anyone who claims to be from "planetminecraft" will be jailed/banned/kicked or whatever you want.

5. protecting- Protect areas like spawn and all around spawn. If you do not protect around spawn then griefers will destroy all around spawn and could build a moat or something. Protect shops, arenas, pixel art and all your other projects.

6. Your own hacker-Having a mod who knows about hacking will help you a lot. And having someone who knows how to code will help even more. Make sure you know they do not use hack anymore and make sure they are VERY trustworthy.

7. Have fun with it! It might be a lot of work to own a successful server might be hard but its pretty fun :) Feel free to post some of your tips in the comments.