From experience (American/Australian/other schools may differ). Here are the "importance" of certain parts of school". This is from experience and I hope it reassures you a little.
As mentioned, this may differ per country, People from England will find this the most beneficial.
Year 3 - ...Seriously nothing to be worried about.
Year 4 - ...Seriously nothing to be worried about.
Year 5 - Okay. You may want to start revising and paying attention for your end of primary school exams. The reason will be explained in a second.
Year 6 - Quite important that you actually pay attention here. whatever your results are dictate what groups you will be placed in at secondary schools. This is basically to get an understanding of your ability.
Start of secondary school
Year 7 to Year 9 - You should be paying some attention now. For the next three years it will be preparing you for exams and placing you in the right ability groups. Don't be worried, this isn't particularly hard. I messed around a fair bit myself in these years.
Year 10 - Okay. as far as I am concerned this is a fairly big year. You will start taking some GCSE preparation as well as picking the subjects you want to do at that level.
Long story short, make sure you listen, learn and importantly don't stress.
(You can still mess around a fair bit though)
Year 11 - Arguably one of the hardest years. This is your moment where everything you have learnt will be judged. (Okay. Seriously. Don't mess around this year).
I am currently in college at the moment but It's a bit far ahead for what I am trying to get across.
The point is, Year 4 or Grade 4 is fairly insignificant in the future, you'll have several more years where you really shouldn't worry about what teachers tell you.
The reason they try and make it a big deal is so that you will achieve your best which lets them have a glance at your abilities.
