53

Down The Rabbit Hole

Collectable

Get Embed Codes

Forum:
HTML:
Link:
avatar
Paril
Developer
Level 80 : Elite Scapegoat
4,415
(I originally wrote this for Facebook; copied it here because I think people might be interested in the kind of stuff that goes on behind the scenes with one half of the development team for PMC and perhaps why things take so long to get done)

Programming is great. There's literally never a shortage of things to do, which I love, but at the same time it feels like an endless rabbit hole where I'll never find the end. I can't finish what I start, and nothing that is finished is ever truly finished. There is always more to do, things to fix, and discoveries that we've left behind in a spur of "we'll fix that later".

So I'm working on the new forums for PMC - am aiming to have them done before the summer ends if everything goes well (that means you too, Cyprezz :p) - which is a long time coming and a huge operation. I admit though I've been trailing far, far behind my expected finish date and I'm not helping by going so deep.

Looking through my logs, it started after I simplified permissions - I added a new permission system to the forums (replacing the overly complex one I had made before) and some new rank-related stuff, and moved some old code around.

This was great, but because of having everything in one spot I had to adjust some stuff so that the caching mechanism was available to all callers who tried to access the moderator list. All good after some finagling, although this wasn't fun as I initially moved everything to one spot and then had to move it somewhere else a bit later.

After skimming around on the things that are done and not done, I realize that the new register/login area needs some immediate attention because it's trailing behind and it should be the #1 priority to finish up first. It's simple and fairly small, so what can go wrong? As I'm working away, I reach the point where I have to implement reCAPTCHA 2.0. Easy, but a bit of a roundabout because I have to remove all of our old reCAPTCHA code and be sure the new stuff works.

After this short detour, my journey lead me back to the forums where I realized that my URLs for new forum threads didn't match our current standard for URLs (it used underscores instead of hyphens, which is what the old forums used and probably why I did that to begin with). After some searching I discovered that not only did we have approximately six different copies (each slightly different, too) of the same function used to generate these names, but because of Cyprezz and I working independently we've accidentally used the wrong versions in some places. This began the deepest programming hole I think I've ever dug myself.

Screw it, I said to myself; instead of frankensteining what is here, I wrote a new routine that acts as a bypass for all of these similar "validate" or "sanitize" operations. This was probably a few hours of work alone just on this function.. but knowing my track record with rewriting things that are already there and then spending days fixing their broken output on the live site, I also wrote a whole unit test that performs some good and bad validations to be sure that each validation type worked in the exact same ways that they did before. There's a whole 9-some hours of work just on making sure dumb people don't put in a letter where we expect numbers.

I can't remember if my next steps were from dreams or picked up on later, but at some point in all of this I also had the bright idea that our caching system needed to be cleaned up a bit. The main problem here is that I didn't have memcache set up still, but I got that working within 30 minutes (and caught a few issues in the process that I hadn't caught prior). I believe this is where I realized that not only were the two interfaces (memory & file caching) inconsistent, but we have a bunch of third party files for a whole file-based caching system that could be condensed down quite heavily (the final, from scratch implementation ended up being only 114 lines, a fraction of Cache_Lite which is what was used before). After writing a brand new interface for the cache that matches our other newer systems (proper object-oriented stuff), I also had to go through every instance of caching being used and fix the calls up so that they matched the new format.

I did have one pretty bad "waking nightmare" though, laying in bed and thinking through how this is all going to work when Cyp and I merge our two very separate codebases back together and have to deal with so many old functions missing in his code. I went back through my history and painstakingly added deprecated function stubs where the old methods were that redirected to the new function automatically, so that at least when we do a merge his code will still work and then we can fix up the deprecated entries later on. I know that despite going through all of that, there are some functions I still forgot to add and I dread having to play hide and seek with those things. At the very least, I can say that the cache system works great and is probably the only chapter listed here that is finally closed.

I went back to where I was before - sanitization. So, this class also involves a routine for verifying/sanitizing larger inputs (like texts that people write), which requires a separate method because it has a lot of extra stuff that we pipe out of it; within this code we have a lot of older stuff that is no longer required, as well as about four separately-hardcoded blacklists (and whitelists) for URLs/spam words, which also merges with the blacklist that is on our database that mods can manage. I've been actually meaning to do this bit for a while, so I figure it's a pretty easy job, no biggie. I finally finished this mid day today, a new database table separate from the old one that contains a URL blacklist, whitelist, ad-link list, and spam words, all of which can be accessed independently from the code, as well as fixing up our other implementations to use this new list code.

Adding these new functions also means that I have to go deprecate some old ones, specifically in the console where the moderators access the blacklist. I started writing the console command, and thought back to the moderators bugging me about the console & how complicated it can be, how annoying it is for me to write new console commands, and then a vision popped in my head of a grammar parser with automatically-generated manual entries that would be easier to read for them. I haven't started on this yet, but the concept is forming and I think I can knock that out in a day or so assuming it all goes well... not looking forward to adjusting all of the old commands to fit the new system though.

So, long story short, ~3500 added lines and ~1200 removed lines in two days, not including the old third party stuff that I was able to delete entirely. It's hard to open one task without also opening several other subtasks that either need to be done or the passion just strikes to need to do while you've got it in your head.. and I also spent an hour ranting here about this. I don't know why. Too much to do, not enough time!

...also I started this whole thing working on the login/register system, which still isn't done. Hopefully when I unravel all of these other things I'll remember where I left off.
Tags:

Comments : 29

Join us to post comments.

Show Comments

1 - 29 of 29

  • jumpyblock
  • Level 43
  • Master Toast
  • January 3, 2017, 3:13 pm
I feel some insane empathy for you right here. I'm not a real programmer with some crazy spaghetti coded website, but I understand what you're feeling. I'm developing a super simple game and sometimes I get overwhelmed with how messy and buggy everything is. I find myself spending hours on end just rewriting methods and stuff my code full of comments.

And then sometimes when trying to fix a problem I get so horribly distracted with a million other things I want to do that I full into that programming "rabbit hole" as you phrased it.

I feel compassion for you. I hope your programming experiences keep working out and more importantly I hope you can find peace in knowing that no program will ever be complete. Good luck! You inspire me :)
  • 7REPTILLIAN7
  • Level 25
  • Expert Architect
  • December 10, 2016, 6:15 pm
Pls.
  • Panda
  • Level 47
  • Master Blob
  • October 11, 2016, 3:02 pm
I love it when you talk tech to me :3
  • Paril
  • Developer
  • Level 80
  • Elite Scapegoat
  • October 11, 2016, 11:22 pm
Why did you ask for my help then disappear :(
  • Panda
  • Level 47
  • Master Blob
  • October 11, 2016, 11:39 pm
My bad, my I.T. support lives all the way out in Canada ._.
  • V3X_
  • Level 14
  • Journeyman Skinner
  • August 15, 2016, 6:02 am
even doe i didnt read it seems like some spoopy stuff yo
  • taterman88
  • Level 61
  • High Grandmaster Batman
  • September 19, 2016, 1:16 pm
^^
  • RenegadeRad
  • Level 9
  • Apprentice Explorer
  • May 28, 2016, 8:49 am
Out of topic, why is the community so dead nowdays? Everyone I knew is dead...
  • Paril
  • Developer
  • Level 80
  • Elite Scapegoat
  • May 29, 2016, 1:34 pm
People move on.

I am under contract to stay here so unfortunately I can't leave :p
  • PythonEX
  • Level 30
  • Artisan Button Pusher
  • June 20, 2016, 1:38 am
Where's the contract? I want to put my name on it so I can never leave too xD
  • RenegadeRad
  • Level 9
  • Apprentice Explorer
  • June 2, 2016, 12:00 am
Hey, Minecraft is such that people should not move on. The fact is that they have ruined the whole vibe of the game for themselves, kinda in a delusion. It still is a good game, I hope you don't move on :P
I guess they got got bored of skinning or something. (Well, That's what happened to me. Now, I'm never posting xD)
  • trashCAN
  • Level 14
  • Journeyman Fish
  • May 19, 2016, 6:10 pm
i just skimmed through this entire thing lel
  • Josh Crafts
  • Level 11
  • Journeyman Architect
  • May 14, 2016, 1:03 pm
tl;dr
You need to blog more often ok
WHO IS MAX?
  • Paril
  • Developer
  • Level 80
  • Elite Scapegoat
  • May 13, 2016, 1:33 pm
Whoops. I meant to change all of his names to Cyprezz but I missed that one lol
  • YTCereal
  • Level 16
  • Journeyman Network
  • May 14, 2016, 12:28 pm
You had once job.
  • Grimdark
  • Level 50
  • Grandmaster Artist
  • May 17, 2016, 6:23 am
Once job
  • YTCereal
  • Level 16
  • Journeyman Network
  • May 17, 2016, 10:07 am
one
  • Speedy
  • Level 48
  • Master Blob
  • May 12, 2016, 6:32 pm
quit ur job and hand over pmc owner status to me much thank~~~~
  • Midnight
  • Site Moderator
  • Level 40
  • Master Poro
  • May 12, 2016, 7:03 am
TL;Dr version, coding takes time and it's easy to get distracted.
  • NoxNyctores
  • Level 23
  • Expert Explorer
  • May 12, 2016, 12:19 pm
Don't forget that when you squish one bug, 30 more appear.
  • Attemptive
  • Level 1
  • New Explorer
  • May 12, 2016, 7:01 am
no
10th diamond.
  • YTCereal
  • Level 16
  • Journeyman Network
  • May 12, 2016, 2:23 am
ew
  • Pines
  • Level 9
  • Apprentice Network
  • May 12, 2016, 1:51 am
nutshell pls
  • Fox
  • Retired Moderator
  • Level 38
  • Artisan Ladybug
  • May 12, 2016, 12:25 am
ur a leg.
  • RenegadeRad
  • Level 9
  • Apprentice Explorer
  • May 28, 2016, 8:48 am
k sry

1 - 29 of 29

Show Comments

Search

Browse

Site

© planetminecraft.com

Welcome