News, news, news..

Maildir is a huge improvement over mbox for storing local mails of users. Why? I will not go into a long explanation about this, because it is not the scope of this post, but just consider that saving each mail in its own file on the disk (Maildir) opposed to saving all mails in a single file (mbox) is much faster. Not only faster, but it is also much easier to manipulate the files (that are individual mails) on the system. For more details on maildir vs. mbox you can see http://www.courier-mta.org/mbox-vs-maildir/

Now, even if Maildir is much faster for many emails kept on the server, when we reach a huge number of files in a single folder the access times to that folder will be considerably slower. Now I am not talking here about a couple of hundreds of mails, but some huge mailboxes with thousands of mails and huge sizes (over 3-5GB in size). You will be amazed that there are peoples that will do that… They are most probably using IMAP and keeping all their mails on the server, or even POP3 and saving one copy of each mail on the server.

Now since Maildir saves each mail in a separate file, it is much easier than it used to be to manipulate the mails. Everyone can write some simple script to do some cleanup based on its needs. A while ago I have stumbled across this python script that does most of the things I needed to cleanup maildir folders. What features was I looking for?

• ability to archive or delete older mails (configurable age of mails)
• archiving old mails in Maildir format, so they can be still used from the email client.
• usage that can be automated in a cronjob.

cleanup-maildir has all the features that I needed (and some others). So I would recommend this little script for a quick solution to cleanup maildirs. Here are some useful options:
-n, –trial-run
Do not actually touch any files; just say what would be done.
-a, –age=N
Only touch messages older than N days. Default is 14 days.
–archive-folder=F
Use F as the base for constructing archive folders. For example, if F is ‘Archive’, messages from 2004 might be put in the folder ‘Archive.2004′.
-d, –archive-hierarchy-depth=N
Specify number of subfolders in archive hierarchy; 1 is just the year, 2 is year/month (default), 3 is year/month/day.
–maildir-root=F
Specifies folder that contains mail folders. Default is “$HOME/Maildir”.

COMMANDS
archive - move old messages to subfolders based on message date
trash - move old message to trash folder
delete - permanently delete old messages

For all the available options check out the help page (cleanup-maildir –help)

Sample usages

Archive messages in ‘Sent Items’ folder over 30 days old (from the maildir of the running user):

cleanup-maildir --age=30 archive 'Sent Items'

Delete messages in the current maildir of the running user older than 60 days:

cleanup-maildir --age=60 delete ''

Archive mails older than 3 months from a specific user maildir (/home/someuser/.Maildir) and place them in yearly archive folders in the form: Archive.YEAR:
cleanup-maildir --age=90 --archive-folder=Archive --archive-hierarchy-depth=1 --maildir-root='/home/someuser/.Maildir' archive ''

Notes:
- the -n parameter can be very useful when you are familiarizing with the script (it will only show you what it will do without actually touching any file).
- when running the script against a different user maildir the script will create the folders with the running user permission. So if you are archiving the mails for usage also from an email client, you will have to run after that the proper chmod command to fix the permissions.
- after the script finishes you will be presented with some general statistics on what it has done. This looks like:
INFO:cleanup-maildir:Total messages: 51507
INFO:cleanup-maildir:Affected messages: 37611 (moved to archive)
INFO:cleanup-maildir:Untouched messages: 13896
- the script looks inside the mail headers to get the dates and not from the date of the actual files on the system.

While still hoping to release Etch as expected in December, the Debian team has revealed the name of the next Debian version: Lenny. This is from debian-news.net and the actual announce is very funny:

“If you managed to read so far: Congratulations! That means you get to be one of the lucky first 10 people to know the codename for the next Debian release: it will be called Lenny.”

Haha… I’m one of the lucky ones

In Part1 of this post I have explained the problem I had faced, when trying to renew one domain registered with RegisterFly. This is the second part of the post and shows a step by step procedure describing the solution that worked for me and allowed me to move away from registerfly and not loose my domain.

1. Why is this happening?

Registerfly is not explaining to their customer what the real problem is. From my own experiences and findings here is what is happening:

• RegisterFly used to be an ENOM reseller. All this started when they moved away from Enom and become a standalone registrar.

• Earlier this year, anyone renewing their domains on registerfly have seen that instead of the normal instant action this required a two step process. This was actually transferring their domain from Enom to RegisterFly’s own system. This caused some confusion but it was working and myself have renewed a few domains like that without any problems except the longer time to finish and the fact that I had received a transfer request that needed to be approved for the renewal to work.

• Starting around the end of October (presumably October 26th), .COM and .NET domains can be only transferred with an AuthCode, as well as .INFO or BIZ domains. This means that you will need the domain authcode in order to make any domain transfer.

I don’t know (and I don’t really care) what exactly is the problem, but it appears that the introduction of AuthCodes has broken the renewal process of RegisterFly domains still hosted on Enom system. The transfers are failing probably because of no authcode being known by RegisterFly.

2. Is your domain renewal problematic or not?

I have about 30 domain registered with RegisterFly and unfortunately only the one that was expiring soon was problematic. Why? Well if the domain is already in RegisterFly system the renewal will take place as expected in matter of minutes without any problems. The problem appears only on domains that are still in the Enom system and RegisterFly tries to actually transfer them in order to renew the domain. Using a simple whois query on the domain you can see if the domain is problematic or not:

If this says:

Domain Name: yourdomainname
Registrar: REGISTERFLY.COM, INC.
Whois Server: whois.registerfly.com
Referral URL: http://www.registerfly.com

then you are safe and you will not see any problems caused by this (and don’t need to follow up this post at all )

But if this is says:

Domain Name: yourdomainname
Registrar: ENOM, INC.
Whois Server: whois.enom.com
Referral URL: http://www.enom.com

then you are probably in big trouble and this post might help you solve the renewal problem.

3. Try to renew it anyway.

If you see one of your expiring domains in the situation presented above (still in the Enom system) then you should try as soon as possible to renew it. Who knows? maybe they have fixed the problem in the meantime, and you will no longer need to get through all this… But if it fails as shown in Part1 for my domain, read on…

4. Preparing to move away…

The next logical thing is to try to move the domain to some other registrar in order to not loose it. Normally any transfer will add one extra year also to the valability of the domain, but our main concern here is to transfer it safely (as this will fail if you are not following all the required steps presented bellow).
So let’s see what you need to prepare the move:

1. if you are using ProtectFly disable it. ProtectFly has various problems and at this time if I send an email to a protectfly enabled email address it will not be forwarded to my own email address, meaning that I will not receive the transfer notifications.
2. ensure that you have a correct admin contact email address. You will receive here the transfer notifications and need to take actions based on the information received in that email (to approve the transfer).
3. unlock your domain: you will not be able to transfer the domain if it is locked. If you had tried to renew it on RegisterFly and it failed they have already unlocked the domain and left it in that state.

5. Get the domain AuthCode

You will need the AuthCode (also referred to as an authorization code, Authinfo code or an EPP code) in order to have the domain transferred.
Where can we find this information in the RegisterFly interface? Well from what I have found out we can’t… But let’s try it out anyway:

• once you have protectfly disabled, go into the domain management for the domain.
• enter in edit mode in the domain contact information page (this will only be showed in edit mode and not view mode).
• just bellow the billing information you should have the Auth Code. If you don’t have it then read on…

Assuming that there is not AuthCode for the domain in the registerfly interface you could try to call them and ask for it. From my experience this has failed and they said they don’t know the code and they can’t give it to me! (unbelievable, right?). The discussion of course finished with the registerfly slogan: “try again in 24h and maybe it will appear in your account).

The way how I was able to solve this, was calling Enom. I called them and explained that I have a domain expiring and this was registered with their former reseller registerfly. I have asked them if they can’t send me the authcode. I don’t know if this will happen the same for everyone, but the person I spoke from Enom was very nice and professional, and he sent me the authcode (on the admin email of the domain) and while I was still speaking with him (he was of course trying to convince me to move my domains to enom, but I explained that I have already started the process with a different registrar and have paid already for the transfer, and he backed up very polite no longer insisting into this), I already got the email with the code. Unbelievable what a difference from RegisterFly support peoples.

Note: the strangest thing is that after I finished the call, and looked again in the registerfly account the same authcode has appeared there also… very strange, and probably if I was not so scared to lose the domain and started the transfer already the next registerfly renewal probably would have succeeded.

6. Move away…

With all this information I could start the transfer successfully. I approved it, entered the authcode and the process started. I was looking into the status and it stayed for about 2 days in the same state (waiting for authorization from previous registrar).

After 2 days I got an email from registerfly about the transfer out (Domain Transfer Request for domain). If I would have not taken any action the transfer would have completed in about 5 days, but of course I opened the link from the email and approved it at once. The next minute the domain was safely in my new registrar’s account and extended until 2008. Victory!…
After all this fight I was able to renew the domain and lost only my time, my nerves and maybe some other things that I can’t see right now, but the domain was safe after all…

Conclusion: I have written this post hoping that it will help other peoples in similar situations. The lack of information and the horrible registerfly support makes it very hard to solve this problem. Also being on time pressure like I was (assuming from previous experience that the renewal process is something very easy and takes just a few minutes) is not helping at all… This post was not intended to be an anti-registerfly one, but just to help you renew your domain if you are in a similar situation (as probably this is all that matters for you) regardless if this is on registerfly or somewhere else.

Apparently Software Engineering is the number one job in America based on its growth, stress level, and pay.

Check out the top50…

RegisterFly.com, inc. is an ICANN Accredited registrar with over 2 million names registered by customers in all 50 states and 120 countries (from their own sayings).
They are a cheap domain registrar solution, and from my own findings at the time I have chosen to work with them (about 3 years ago), they had the best prices on the domain registration marked. Being a cheap provider I would not expect some stellar technical support, but not even like this…

Peoples that know me at least a little, know that I am normally a very calm person and I will normally not start a post that denigrates others services… I am just to busy to think that it is worth to waste my time with such discussions, and better use my available time to write about something positive that might be useful to others also. But during the last days that I had some terrible problems with RegisterFly while trying to renew one of my friends domain, that just made me so upset and nervous on these incompetent peoples and made me decide to write about this. I assume other peoples are in the same situation and might benefit from my experience because in the end I was able to save the domain and move it elsewhere.

This article is divided into two parts one explaining the reason for this post (showing the problems I have encountered) and the second part showing how I was able to solve this problem and escape from this nightmare that is registerfly renewal system.

In Part1 I will give a very short description of the problem I had and the situation I was facing when trying to renew one of my domains registered with registerfly. This shows the reason for my frustration and the reason to write this post… So here we go…

I have used RegisterFly for my own domains and some friends also, for several years. The reason why I have chosen them was because they were one of the few registrars that were accepting payments from my country (that most of the internet seemed to ban a while ago) and also because of their good prices. Later on I had no problems at all and continued to use them without even thinking to move somewhere else. There was no reason for this as I was not using any other services that I have seen peoples complaining on the internet about them (no hosting, no dns, etc… I am hosting my own DNS and I don’t care about other services then the ones for registering my domains).

Normally when you have one domain expire then you only have to pay for the extension and the change is almost instant (a few minutes and the process is done). You don’t even think that there can be problems there and the only thing that matters is to remember to renew the domain and pay the actual extension on time. This on any registrar but not on registerfly of course… So here is my story:

1. The domain under discussion was one of the domains I am maintaining for one of my best friends. If this would have been my own domain probably the situation would have been different, but I could not loose his domain and have him get upset with me because of registerfly incompetence.

2. I received the email notification on 13-Oct (1 month before the expiation) notifying me that the domain will expire soon. A few days after that when I had some time I went to registerfly interface and tried to renew the domain as I have done many times before. The system took the money from my account and sent me a notification (Registerfly.com - Renewal/Extend domain Request). This was saying that the process has started and the TRANSFER has been initiated. I was not worried at all since I have renewed some domains earlier this year and it happened the same (with the transfer). They have a faq page explaining this… so all looked ok.

3. After a while (2 days approx) I received an email notifying me that the process has failed… Hmm… No error in the mail saying why it failed. At this time I got a little worried but assumed that there are some problems on their system, so I have opened a support ticket explaining what happened. Their response came after one day and looked like a standard reply sending me to their faq page. I replied and said I have read that, but the process has failed. After a while again their reply came and it was basically saying that they have some problems with their renewal system and I should try again in a few days (24h-72h). Ok, I can understand that… we all have problems and this was not pressuring… So I let it be and marked it as something I had to return and have it done in a few days.

4. I was obviously not considering what a mess this company is… So I got back to my works and being involved in some major projects I have left it untouched for a while. On the 1-Nov I have remembered to try it again. So again I started the process to renew the domain. I forgot to mention that they refunded properly my money at the failed renewal so this was ok. So basically I have done again the renewal process and it was exactly the same (send me a notification, linking to their faq about the transfer, etc.). After a while (1 day) I received the same failure message… Hmm… This time I was getting nervous… how can they still have problems after so many days I was wondering? So I decided to call them and see what is the problem.

5. Anyone that has called RegisterFly support knows what comes next… When you call them the answering machine greets you with an incredible message (for ex: you are caller no. 14 waiting in line, and the estimated time of waiting is 64mins)… ouch… and they don’t have toll-free number… so waiting is costing money… Anyway after many tries at different hours of the day I finally was able to get someone on the phone. I asked what is the problem, and the reply was that they have some problems with their renewal system and I should try again in a few days (the way they say that is something like: “Sir, please try again in 24-72h). I explained that I have tried again 2 weeks ago and got the same answer and I am getting nervous of loosing the domain, and he responded that I don’t have to worry and I should try again. I was trying to see if he can’t force the process and he said sure… I will do that for you sir.

6. After getting off the phone I have received the same email notification I got already 2 times before and this appeared as I have done that myself. But I said to myself maybe they have done it in a different way… But after another day I got again the failure message.

7. To keep the story as short as possible in the next days I have tried this again: failed renewals attempts, calling them again, tickets, etc. The result was the same: there were only a few days left to renew the domain and nothing changed: failed renewals/transfers and the same replies (try again in 24h).

At this point I have already searched the Internet for other peoples with similar problems and I was horrified on my findings… Just look at a few links:
Registerfly Gripe Site
My domain in RegisterFly can not renew and transfer
Registerfly, such crap!, etc… (just google it)
showing that other peoples had similar experiences. Peoples were loosing their domains because of registerfly incompetence and some even lost their business because of this..

With the help from the information gathered from other online resources and lot of determination I was able in the end to renew the domain. In Part2 of this post I will describe a step by step solution that worked for me.

The release of the next stable Debian version, 4.0 (code Etch) is getting closer. Peoples following the debian testing branch have noticed probably already that the latest base-files update has changed their system banner and instead of the old:

/etc/issue
Debian GNU/Linux testing/unstable \n \l

after this update it now shows:

/etc/issue
Debian GNU/Linux 4.0 \n \l

and

cat /etc/debian_version
4.0

Here is the update output:
base-files/etch upgradeable from 3.1.16 to 4:

Preparing to replace base-files 3.1.16 (using .../base-files_4_powerpc.deb) ...
Unpacking replacement base-files ...
Setting up base-files (4) ...
Installing new version of config file /etc/debian_version ...
Installing new version of config file /etc/issue ...
Installing new version of config file /etc/issue.net ...

Lighttpd is a small and fast webserver developed with security in mind and a lot of features. Until recently Debian packages were available only in the unstable release, but now they have been included in Debian Etch (testing) and will most certainly make it to the next stable Debian release (etch is scheduled to be released in December).

Myself I have been using Lighty for a while, either compiled from sources or using the unstable debian package that worked very well for me without any special problems. Though I am using it only to serve static content (images mainly) and not use any other special features. In this case lighttpd outperforms apache in my experiences by far in performance (requests / second) and also resources utilizations (CPU and memory).

So what do we have available in latest debian testing update? We have of course the latest stable version 1.4.13 released on 2006-10-09.

aptitude show lighttpd
Package: lighttpd
New: yes
State: installed
Automatically installed: no
Version: 1.4.13-4
Priority: optional
Section: web
Maintainer: Debian lighttpd maintainers

Uncompressed Size: 750k
Depends: libattr1 (>= 2.4.4-1), libbz2-1.0, libc6 (>= 2.3.6-6), libldap2 (>= 2.1.17-1), libpcre3 (>= 4.5),
libssl0.9.8 (>= 0.9.8c-1), zlib1g (>= 1:1.2.1), lsb-base (>= 3.0-3), mime-support
Recommends: php4-cgi | php5-cgi
Suggests: openssl, rrdtool, apache2-utils
Provides: httpd, httpd-cgi
Description: A fast webserver with minimal memory footprint
lighttpd is a small webserver and fast webserver developed with security in mind and a lot of features. It has
support for
* CGI, FastCGI and SSI
* virtual hosts
* URL rewriting
* authentication (plain files, htpasswd, ldap)
* transparent content compression
* conditional configuration
and configuration is straight-forward and easy.

Homepage: http://www.lighttpd.net

Also besides the main package there are other additions available:

i   lighttpd   - A fast webserver with minimal memory footprint
p   lighttpd-doc   - Documentation for lighttpd
p   lighttpd-mod-cml   - Cache meta language module for lighttpd
p   lighttpd-mod-magnet   - Control the request handling module for lighttpd
p   lighttpd-mod-mysql-vhost   - MySQL-based virtual host configuration for lighttpd
p   lighttpd-mod-trigger-b4-dl   - Anti-deep-linking module for lighttpd
p   lighttpd-mod-webdav   - WebDAV module for lighttpd

The configuration file consists from one main file: /etc/lighttpd/lighttpd.conf and also some extra configuration files for some modules:

/etc/lighttpd/conf-available/
10-auth.conf
10-cgi.conf
10-cml.conf
10-fastcgi.conf
10-proxy.conf
10-simple-vhost.conf
10-ssi.conf
10-ssl.conf
10-trigger-b4-dl.conf
10-userdir.conf

Enabling the extra modules can be done using lighty-enable-mod and disable with lighty-disable-mod scripts, or this can be done manually by creating symlinks from conf-available/ to conf-enabled/. For example to enable fastcgi (that most people using lighty with php will probably want to enable ) you just have to run:

lighty-enable-mod fastcgi

As with the apache a2enmod/a2dismod running the script without any parameters will show the available choices (modules that can be enabled/disabled):

lighty-enable-mod
Available modules: auth cgi cml fastcgi proxy simple-vhost ssi ssl trigger-b4-dl userdir
Already enabled modules:
Enable module: 

Besides the extra modules, lighttpd comes with the following modules available (and enabled in the default configuration file):

• enabled by default: mod_access, mod_alias, mod_accesslog
• disabled by default: mod_rewrite, mod_redirect, mod_status, mod_evhost, mod_compress, mod_usertrack, mod_rrdtool, mod_webdav, mod_expire, mod_flv_streaming, mod_evasive

From /etc/lighttpd/lighttpd.conf:

server.modules              = (
"mod_access",
"mod_alias",
"mod_accesslog",
#           "mod_rewrite",
#           "mod_redirect",
#           "mod_status",
#           "mod_evhost",
#           "mod_compress",
#           "mod_usertrack",
#           "mod_rrdtool",
#           "mod_webdav",
#           "mod_expire",
#           "mod_flv_streaming",
#           "mod_evasive"
)

To enable one of the other modules just uncomment the particular module and define the proper configurations for it, if it is the case (use the lighttpd documentation for more details about each module), and restart lighty:

/etc/init.d/lighttpd restart

I hope you enjoyed this post, that wanted to introduce this great web server software and the particularity of the debian package and its specific tools and default configurations.