We had a small problem. Although transperant to all users (in fact, it didn't even cause any real delay in updating), I figured it's best to report about it here so everyone knows...

As of some time in the last 12 to 24 hours, Safari Plugins Repo plist file went offline. I don't know when it happened, or why it happened, but it did. Instead of throwing standard 404 error or alike, it redirected to the author's blog page. Again, I have no idea why it happened, but it did... As result of that, the crawler some decided to set off a security flag to the server's security watch (variant of mod_security + SuExec), and cause the crawler to STOP WORKING on the spot, litterally. For the record, I did have code in the crawler to "skip" problematic sources, so I still don't know why this particular output threw the security alert...

When I first noticed the problem (not knowing what it was), I thought some how the scheduled task went wonky, so I checked into that, and didn't fix it. So I tried several different things, including recompiling php... Appearently, none of them fixed the problem. Eventually, I decided to track down using temp files to see what data I'm getting and was finally able to locate the problem. On the bright side though, there was no software updates/additions in the past how many ever hours it had problem, so nothing was "missed" for those that wants to keep super-dooper-up-to-date with your packages. For now, I have removed the source from repo.sc crawler, so it is no longer being updated. The packages offered by the source is still in our system, I haven't had the chance to test it manually to see whether or not they still work. I'll test it and remove them if they no longer works.

I'll be going back to revisit the crawler again later (after I add the much requested "add category" feature), and see if I can find a way to prevent this from happening... so we don't have to deal with this in the future.

Cheers.

ok, cool, would it be possible if the crawler also stated "currently offline repo's" or something like that? the ones it skips? it could show them on one page so that we know what ones are working atm or whatever?

Dont worry if its not possible, its cool