TightURL Development Blog

The coming plugin system will finally allow for a long-planned feature.  Custom URL “shorteners”, which I’ve been calling squisher engines, or just squishers.

For example, one user requested the ability to generate easily remembered URLs based on common words.  This will be able to be implemented as a squisher engine plugin, and installed into your TightURL installation.

As a result of this new feature, TightURL will also support allowing the user to choose the squisher to be used.  Site administrators will be able to choose one or more squishers to allow users to choose from.

Site admins who know some PHP (or hire someone who does) can easily create new squisher engines without extensive study or modification of the TightURL code.  Squisher engines receive the URL, submitter’s IP and submitter’s User-Agent, and they return either a shortened URL or an error code.

Been sick, nothing much happened for a couple of weeks. Then I picked up some code I had stashed in the “next version” folder, which like most of the code I find, had some problems. After the cron debacle I wasn’t sure I wanted to put much time into it, but it turned out there were two lines of code I had to change and now I have a working implementation of hooks and filters and I only spent a few hours on it.

Hooks and filters? I don’t know if they were the first, but it’s basically the same system used by WordPress to allow extensibility since about version 1.5. Hooks are places in the code where plugins can interact with TightURL. Filters are pretty much the same thing, except specifically intended to allow plugins to alter something, usually output displayed to the user.

What exactly does this mean?
Initially, I plan on adding three hooks to TightURL. ‘Squish’, ‘Squisher’, and ‘Squished’. Squish will run prior to shortening a URL and will be passed the URL to shorten, the IP of the submitter, and the User-Agent from the browser.   The Squish hook allows for the creation of anti-abuse/policy plugins.

Squisher will run instead of the “native” URL shortener.  It receives the URL to shorten, IP, and User-Agent.  The Squisher hook allows for the creation of new URL shortening engines.  Squishers will return the shortened URL.

Squished  will run immediately following URL shortening.   It receives the URL, the output from whatever squisher ran, the IP, and the User-Agent. The Squished hook allows for the creation of post-shortening tasks.

Developers should take care, as all three hooks lengthen the amount of time it takes for your TightURL installation to return a shortened URL.  I have found even slight delays generate complaints.  In particular, resist the temptation to create complex Squish plugins, or Squished plugins that perform maintenance tasks.  Maintenance tasks should be run by the the internal job scheduling system, and complex, involved, pre-shortening processing simply is not viable.  You may be able to convert this instead into a small bit of code that runs with the Squish hook, and the rest runs as a scheduled job.  And finally, Squishers as well, need to run quickly and return a result to the user.  As with the Squish and Squished hooks, you should use the job scheduler for processing that takes more than a fraction of a second to run.

I’ve done some work in both the future major release, and in the production version. The next version released will be known as 0.1.3.6, it will be based on the version that runs at tighturl.com, and I am going to add some new functionality before release. I don’t think it will take me too long.

The problem I want to solve, is that right now you have to edit one of the config files full of regexes to blacklist a site. From time to time I make an error there that ends up disabling the service, and users disappear. There are some other concerns, but the main thing is, you shouldn’t be editing files all the time. I need to validate the regexes, but most importantly, I need to store that data in the database, and once I’m doing that, the “killbot” will also be able to blacklist the domains hosting the links it disables. I’ll have to give you a way to enter and remove things from a password-protected web page too.

I think this is important enough to hold the release for. I’m trying to work on TightURL at least an hour every day for now.

As promised earlier, I’m investigating an interim release of TightURL so I can continue work on the major release, which I need to spend a bit more time with before I can even run it myself anywhere. It’s really still development code.

I don’t think it will be too difficult to get the interim release out, but it also doesn’t really do much the available version doesn’t do, with the exception of bundling the “killbot” for those who have cron access. Nobody should run a public URL shortener without it.

I’ve done a little work on the new additions report I get every hour to include some statistics. It now shows the total records added on today’s date along with the total records added today that have been disabled. The midnight (00:00) report includes totals for yesterday.

I need to convert the hardcoded domain filter I have now into something user-configurable.

The additions report will be part of the next major version of TightURL as a task run by the job scheduler. It’s (been) available on request to those who can run cron jobs.

I recently added the Spamhaus DBL to TightURL’s blocklist checking code. Shortly after, I discovered tighturl.com is on the DBL. We’ve at about day 17, and at least a half dozen inquiries gone unanswered.

This will affect deliverability of the support and developer’s mailing lists. Most importantly, it reflects poorly on Spamhaus that it is run in such an irresponsible manner. The fight against abuse is taken far more seriously by me, the sole developer of TightURL than it is by Spamhaus, a business that takes in money and whose data is relied upon by so many.

I have been setting up my “new” development system. I’m going back to developing TightURL under Linux. I can’t say that there’s been any compatibility issues that came up while I was developing under Windows, so it is likely that TightURL will continue to be compatible with WAMP (Windows, Apache, MySQL, PHP) stacks.

EDIT: 24 days, no response from Spamhaus. Apparently the lights are on, but nobody’s home.

Just to update those who are interested in this sort of thing…

Work continues on a new release. This will be a major release that incorporates a number of things that needed to be done. How many of these things get done remains to be seen, but I’m trying not to get bogged down with too many things being done at the same time. So far, the changes of note and work under active development are:

• PHP 5.0 now required
• Black/White listing moved to the database
• Cron-like task scheduling and execution
• Scheduled new additions report
• Scheduled anti-abuse detector
• Abuse mail processing script

Things likely to be included:

• TightURL Library
• Localization
• New release and security notifications

Things that would be nice but aren’t yet likely to be in the next release:

• Improved templating
• Improved API

Input from users of both tighturl.com and the TightURL source code is welcome. The TightURL Library is an attempt to separate the front-end of TightURL from the back-end, making it easier to create your own URL shortening front-ends that use TightURL’s library to shorten URLs, handle abuse, etc.

I know many of you are interested in an improved API, but so far I haven’t found a simple way to do this or any way to do it that doesn’t require me to shoehorn TightURL into some framework that’s many times the size of TightURL’s code itself. I’m still hoping to figure out a simple way to add a REST API.

I’ve avoided doing anything to make TightURL require PHP 5 up until now, but PHP 5 is no longer “new” by any stretch of the imagination, and I need to make use of code that requires PHP 5.

So there you have it. The next major release of TightURL will require at least PHP 5.0. For the time being, I’m going to try not to force the required version up to PHP 5.3.

Although I’ve got an awful lot of other things in my life that I have to take care of, admittedly much of the latest delay is due to getting a little thrown off TightURL development by the discovery that I’d sank so much time into a non-working PHP cron system I found in Google. I can report now though that I’ve replaced the broken code with something that works correctly, and it passes tests.

I’m back to the point now where I was when I discovered the other code didn’t work. That means a little integration work and a little testing of the job processor, and I can finally get away from a major, if necessary, distraction.

The next thing I have to get back to work on is the blacklisting system. It’s been half-written since January, when I realized I was going to need the cron system for the blacklisting system, and stopped the blacklisting system development to quickly (ha ha ha) bang out the cron code.

Every time I write something to read e-mails, I forget about quoted printable. I have some kind of mental block about it. It is the bane of my e-mail reading existence. I got abuse mail from Engadget the other day. First I was surprised they use Google for mail. I don’t know why this still surprises me, but I find it hard to take a company seriously who does this with their mail. It tells me that what something’s perceived cost is, is really the only thing that matters to them. Second, the mail contained a Quoted Printable part. My poor regex was laid low by Quoted Printable.

What’s this in regard to? A future release of TightURL will include a script that analyzes inbound abuse complaints by e-mail and replies to the sender when appropriate. Sadly after realizing it was going to be a major undertaking, and looking around for acceptable PHP code to properly parse the complicated mish-mash that is our beloved Internet mail, I ended up with another class from PHPClasses, 80K of BSD licensed PHP code to parse an e-mail. It continues to bother me that frequently for the sake of something like parsing an e-mail, something which used to be pretty simple, I end up with a class or library that’s several times the size of the script that uses it.