(I originally wrote this for Facebook; copied it here because I think people might be interested in the kind of stuff that goes on behind the scenes with one half of the development team for PMC and perhaps why things take so long to get done)

Programming is great. There's literally never a shortage of things to do, which I love, but at the same time it feels like an endless rabbit hole where I'll never find the end. I can't finish what I start, and nothing that is finished is ever truly finished. There is always more to do, things to fix, and discoveries that we've left behind in a spur of "we'll fix that later".

So I'm working on the new forums for PMC - am aiming to have them done before the summer ends if everything goes well (that means you too, Cyprezz :p) - which is a long time coming and a huge operation. I admit though I've been trailing far, far behind my expected finish date and I'm not helping by going so deep.

Looking through my logs, it started after I simplified permissions - I added a new permission system to the forums (replacing the overly complex one I had made before) and some new rank-related stuff, and moved some old code around.

This was great, but because of having everything in one spot I had to adjust some stuff so that the caching mechanism was available to all callers who tried to access the moderator list. All good after some finagling, although this wasn't fun as I initially moved everything to one spot and then had to move it somewhere else a bit later.

After skimming around on the things that are done and not done, I realize that the new register/login area needs some immediate attention because it's trailing behind and it should be the #1 priority to finish up first. It's simple and fairly small, so what can go wrong? As I'm working away, I reach the point where I have to implement reCAPTCHA 2.0. Easy, but a bit of a roundabout because I have to remove all of our old reCAPTCHA code and be sure the new stuff works.

After this short detour, my journey lead me back to the forums where I realized that my URLs for new forum threads didn't match our current standard for URLs (it used underscores instead of hyphens, which is what the old forums used and probably why I did that to begin with). After some searching I discovered that not only did we have approximately six different copies (each slightly different, too) of the same function used to generate these names, but because of Cyprezz and I working independently we've accidentally used the wrong versions in some places. This began the deepest programming hole I think I've ever dug myself.

Screw it, I said to myself; instead of frankensteining what is here, I wrote a new routine that acts as a bypass for all of these similar "validate" or "sanitize" operations. This was probably a few hours of work alone just on this function.. but knowing my track record with rewriting things that are already there and then spending days fixing their broken output on the live site, I also wrote a whole unit test that performs some good and bad validations to be sure that each validation type worked in the exact same ways that they did before. There's a whole 9-some hours of work just on making sure dumb people don't put in a letter where we expect numbers.

I can't remember if my next steps were from dreams or picked up on later, but at some point in all of this I also had the bright idea that our caching system needed to be cleaned up a bit. The main problem here is that I didn't have memcache set up still, but I got that working within 30 minutes (and caught a few issues in the process that I hadn't caught prior). I believe this is where I realized that not only were the two interfaces (memory & file caching) inconsistent, but we have a bunch of third party files for a whole file-based caching system that could be condensed down quite heavily (the final, from scratch implementation ended up being only 114 lines, a fraction of Cache_Lite which is what was used before). After writing a brand new interface for the cache that matches our other newer systems (proper object-oriented stuff), I also had to go through every instance of caching being used and fix the calls up so that they matched the new format.

I did have one pretty bad "waking nightmare" though, laying in bed and thinking through how this is all going to work when Cyp and I merge our two very separate codebases back together and have to deal with so many old functions missing in his code. I went back through my history and painstakingly added deprecated function stubs where the old methods were that redirected to the new function automatically, so that at least when we do a merge his code will still work and then we can fix up the deprecated entries later on. I know that despite going through all of that, there are some functions I still forgot to add and I dread having to play hide and seek with those things. At the very least, I can say that the cache system works great and is probably the only chapter listed here that is finally closed.

I went back to where I was before - sanitization. So, this class also involves a routine for verifying/sanitizing larger inputs (like texts that people write), which requires a separate method because it has a lot of extra stuff that we pipe out of it; within this code we have a lot of older stuff that is no longer required, as well as about four separately-hardcoded blacklists (and whitelists) for URLs/spam words, which also merges with the blacklist that is on our database that mods can manage. I've been actually meaning to do this bit for a while, so I figure it's a pretty easy job, no biggie. I finally finished this mid day today, a new database table separate from the old one that contains a URL blacklist, whitelist, ad-link list, and spam words, all of which can be accessed independently from the code, as well as fixing up our other implementations to use this new list code.

Adding these new functions also means that I have to go deprecate some old ones, specifically in the console where the moderators access the blacklist. I started writing the console command, and thought back to the moderators bugging me about the console & how complicated it can be, how annoying it is for me to write new console commands, and then a vision popped in my head of a grammar parser with automatically-generated manual entries that would be easier to read for them. I haven't started on this yet, but the concept is forming and I think I can knock that out in a day or so assuming it all goes well... not looking forward to adjusting all of the old commands to fit the new system though.

So, long story short, ~3500 added lines and ~1200 removed lines in two days, not including the old third party stuff that I was able to delete entirely. It's hard to open one task without also opening several other subtasks that either need to be done or the passion just strikes to need to do while you've got it in your head.. and I also spent an hour ranting here about this. I don't know why. Too much to do, not enough time!

...also I started this whole thing working on the login/register system, which still isn't done. Hopefully when I unravel all of these other things I'll remember where I left off.