The enhancements I’ve mentioned previously have worked so well, I was unaware of the latest abuse attack for several days.  The system at present is too crude and incomplete to share yet, but I hope to share it as a continuation of the release version of TightURL.  I don’t know if I’ve got a PHP 5.x dependency yet.   I hope everyone’s running on PHP 5 by now…

After 78 days (seventy eight days!), dozens of inquiries and no response from them whatsoever, Spamhaus took tighturl.com off their DBL.   You really don’t want to expose your mail to the mercy of Spamhaus.  I recommend anyone running a public URL shortener to get a domain you only use for the URL shortener.  The Spamhaus listing only made my users suffer since the only mail I send from tighturl.com is abuse@tighturl.com and the support and developers mailing lists.   I don’t think Spamhaus cares about the irony.

So now I’m adding an audit log.   The need for this became apparent during initial development, but again, necessity forced it into being.

It’s apparent that the blocklisting is going to take up my next available block of time, but it’s important to get that done, and done correctly.

It’s also important to stress again, that running a public URL shortener requires constant monitoring.  I’m trying to make it so that it’s as small a time commitment as possible, but if you want to run a URL shortener that accepts new URLs from the public you have to be reachable by e-mail most of the day, every single day, in case your URL shortener or your abuse address or your upstream provider needs to tell you something.

I have moved (again).  This is very disruptive, and thus I haven’t done a lick of work on TightURL since.

That doesn’t mean I didn’t do anything before I moved though.  I have finally gotten into the thick of things as far as moving the blocklist data out of PHP and into MySQL is concerned.  As is so often the case with TightURL, abuse taking place over at tighturl.com required immediate action, and thus the beginnings of the new, much more sophisticated blocklisting system were laid down in the production version.  Unfortunately that also means replicating that code in the “new” version and the “next” (based on the production code) version.  I’ve got it into the “new” version, but not yet into “next”.

At this time, I have no UI for the black and white listing functionality.  I will get that into the “new” version.  Whether or not it gets into the “next” version is an open question right now.

Bad Behavior 2.2.3

Anyone who upgraded their Bad Behavior using the updated Bad Behavior for TightURL connector should *NOT* upgrade to Bad Behavior 2.2.3 .  Unfortunately 2.2.3 contains a very rare API change that I will have to make in the connector before you can upgrade.  I’ll announce it when this is ready.

UPDATE: I read the update notice for Bad Behavior a little too quickly.  You can continue upgrading Bad Behavior.

and http://fatfree.sourceforge.net/page/routing-engine/representational-state-transfer-rest-and-classes-objects

I only just discovered both of these, but either looks promising.

Since Fat Free Framework shows me procedural PHP examples, that’s a plus in its favor.

However, since we’re probably going to need an API-key based system, I’m going to need to get an Admin section up that includes user accounts and user profiles.  That also ties into the question of templates.  I really want to make a final decision about a new template system (or not, and which to use if so) before then so the admin section can be templated also.

Fat Free Framework as it happens, has a template system.  That said, I don’t want to make TightURL itself dependent on any framework, even if it’s tiny.

Which brings up another matter… I haven’t decided how to connect the API to TightURL.  My original thought was to keep everything in the same namespace, such that you would have something like:

http://example.com/api

vs.:

http://api.example.com

One of the original intents behind TightURL was for people to run small-scale and/or private URL shorteners.  The first approach probably works best for those people since they may not be able to add subdomains.  The second approach allows you to split API traffic off from the main web site.

Although I like the “cleanness” of the first approach (and that’s an entirely subjective opinion), TightURL can no longer be run on the simplest hosting setups.   It’s going to require PHP 5.3 or newer, and it already needs to be able to do DNS lookups and open network connections to fetch the contents of URLs.  This precludes really low end or free hosting.  I’m not exactly bothered that you can’t run TightURL on free hosting, given that virtually all the URLs submitted to tighturl.com from free hosting services are abusive links.

In the meantime, I am going to be porting the TightURL interface for Bad Behavior to the Bad Behavior 2.2 series.  This will be made available to all TightURL users (current version and “next major version”).

EDIT (2012-03-02 21:58 EST): Now available!

For example, one user requested the ability to generate easily remembered URLs based on common words.  This will be able to be implemented as a squisher engine plugin, and installed into your TightURL installation.

As a result of this new feature, TightURL will also support allowing the user to choose the squisher to be used.  Site administrators will be able to choose one or more squishers to allow users to choose from.

Site admins who know some PHP (or hire someone who does) can easily create new squisher engines without extensive study or modification of the TightURL code.  Squisher engines receive the URL, submitter’s IP and submitter’s User-Agent, and they return either a shortened URL or an error code.

Hooks and filters? I don’t know if they were the first, but it’s basically the same system used by WordPress to allow extensibility since about version 1.5. Hooks are places in the code where plugins can interact with TightURL. Filters are pretty much the same thing, except specifically intended to allow plugins to alter something, usually output displayed to the user.

What exactly does this mean?
Initially, I plan on adding three hooks to TightURL. ‘Squish’, ‘Squisher’, and ‘Squished’. Squish will run prior to shortening a URL and will be passed the URL to shorten, the IP of the submitter, and the User-Agent from the browser.   The Squish hook allows for the creation of anti-abuse/policy plugins.

Squisher will run instead of the “native” URL shortener.  It receives the URL to shorten, IP, and User-Agent.  The Squisher hook allows for the creation of new URL shortening engines.  Squishers will return the shortened URL.

Squished  will run immediately following URL shortening.   It receives the URL, the output from whatever squisher ran, the IP, and the User-Agent. The Squished hook allows for the creation of post-shortening tasks.

Developers should take care, as all three hooks lengthen the amount of time it takes for your TightURL installation to return a shortened URL.  I have found even slight delays generate complaints.  In particular, resist the temptation to create complex Squish plugins, or Squished plugins that perform maintenance tasks.  Maintenance tasks should be run by the the internal job scheduling system, and complex, involved, pre-shortening processing simply is not viable.  You may be able to convert this instead into a small bit of code that runs with the Squish hook, and the rest runs as a scheduled job.  And finally, Squishers as well, need to run quickly and return a result to the user.  As with the Squish and Squished hooks, you should use the job scheduler for processing that takes more than a fraction of a second to run.

The problem I want to solve, is that right now you have to edit one of the config files full of regexes to blacklist a site. From time to time I make an error there that ends up disabling the service, and users disappear. There are some other concerns, but the main thing is, you shouldn’t be editing files all the time. I need to validate the regexes, but most importantly, I need to store that data in the database, and once I’m doing that, the “killbot” will also be able to blacklist the domains hosting the links it disables. I’ll have to give you a way to enter and remove things from a password-protected web page too.

I think this is important enough to hold the release for. I’m trying to work on TightURL at least an hour every day for now.

I don’t think it will be too difficult to get the interim release out, but it also doesn’t really do much the available version doesn’t do, with the exception of bundling the “killbot” for those who have cron access. Nobody should run a public URL shortener without it.

I need to convert the hardcoded domain filter I have now into something user-configurable.

The additions report will be part of the next major version of TightURL as a task run by the job scheduler. It’s (been) available on request to those who can run cron jobs.