Wow…. Let me just say that I have typically been inundated with junk mail on my primary address. It’s associated with this domain and has been hosted in an older sendmail setup for quite some time (not really by choice, but because that’s what was installed on the old vps.) I’m currently migrating to postfix which I’m a bit more familiar with and had used successfully to at least tag my junk mail as SPAM for sorting at home. Well, my older postfix system didn’t have too many options to add on and tweak and so this weekend I’ve invested some time in migrating the main mail server over to a newer postfix install with amavis and postgrey and a few other tools.

What is postgrey? It’s basically a greylisting tool for postfix. It essentially decides whether or not the mail server accepts a message based on a simple rule – “if I’ve seen the ip address/sender/recipient a couple times and it’s been more than 5 minutes between seeing those, it’s probably a legitimate mailserver.” By the same token “if I’ve seen the same triplet in immediate succession it’s not standards compliant and therefore likely a spammer.” Postgrey supports whitelisting if you’re not patient enough to let it build it’s own whitelist.

You should note that it doesn’t specifically reject messages, it simply says try later. If the sending mailserver is standards compliant and waits and then retries a bit later, and then is patient and well behaved and tries again…. the message arrives. If your mailserver just tries to blast the other addresses at your domain then it’s likely a spammer and is consistently asked to wait and try later.

I was really not expecting a great improvement in the volume of junk mail. I’ve used a number of tools in the past from blacklists to making the mailserver wait for 30 seconds before answering, dns tricks, etc. For the most part each of those methods has cut down a bit on the junk, but not put a huge dent in the volume.

I have been pleasantly surprised. Over the last 3 days I have received a grand total of probably 5-6 junk messages. (A few more have sneaked past the OLD mail server which is still configured as an mx at the moment until I’m sure all of my transition is setup.) What would I normally receive…. it’s hard to put an exact number but I would expect about 150 or so junk messages in those days.

So – how is it setup?

For ubuntu it was quite simple – apt-get install postgrey brings in all of it’s dependencies and sets it to run by default as a service. In postfix’s main.cf you need to add the following into your smtpd_recipient_restrictions…. check_policy_service inet:127.0.0.1:10023 (Of course you’ll need to save the changes and reload postfix to have that take effect. Oh, make sure that postgrey is running too.)

450 “try again later” is the official response that postgrey returns. The logic here is that spammers get lot’s of “error/address not found/bounce/shut up and go away spammer” messages and it’s not worth their time to try to get a few more messages delivered by making a mailer that gracefully handles all of these (or one of these.) Hopefully that holds true for quite some time because it’s been a fantastic improvement here.

Some other benefits of using postgrey. Spam messages are stopped before the message is officially accepted. So…. no bounces (backscatter spam), no cpu overhead scanning for viruses, scanning for and tagging as spam, no dns searches to try to validate senders/etc…. So, lower cpu overhead, marginally lower bandwidth.

Just a quick note for what is really a simple fix. I have been using rsnapshot to back up several machines lately. If you haven’t taken a look at rsnapshot on linux for managing snapshot backups (where only the changes get backed up after the initial run….) it’s definitely worth a look.

Anyway, I had recently added a new system to be backed up and was seeing the following error from my Rsnapshot log…. ERROR: /usr/bin/rsync returned 127 while processing hostname….

The first week it happened I postponed looking into it, but this week when it happened a light bulb went on. Maybe I don’t have rsync installed on the remote system. Logged in and… guess what $rsync command not found….

The moral of the story is that rsnapshot uses rsync and needs it to be installed both on the system that is serving as the host for backups and any systems that are being backed up.

This post is going to be somewhat of a “link dump” for me of some pages that I’ve been perusing lately. After playing with RT (request tracker) – I added a few ticket items for the home network. Now, if you’ve been a longtime reader and sorted through ALL of these posts here you’ll know that I’ve made use of software raid on the home systems. Why? Linux software raid seems fairly reliable (so far – 2 years +). It doesn’t depend on a specific piece of hardware. In short IF the worst happens and the array fails I should be able to retrieve data from an individual drive more easily than if it were hardware raid. I’m using Raid Level 1 (cloning/mirroring) and ext3 is the filesystem on top. I’ve had some slight problems with one drive in both the Desktop and server arrays and both arrays had been degraded for some time. My goal was ultimately redundancy and to eliminate the disruption that hard drive failures have given over the last few years.

When push comes to shove I’m cheap and hadn’t looked at replacing the drives yet. (Potentially a problem especially since I was down to 1 drive for 2 out of 3 arrays.) So, I was running a risk for several months. The main reasons I’ve procrastinated is 1) I’m cheap and 2) I’m busy. The other reason though is the two arrays were made of 400gb sata drives which are now hard to find. 500 gb is the new standard size. Now, I don’t want to waste space so… it’s been a bit of a mental puzzle thinking about replacing one, growing the partition size and then replacing the other to pass along to the second system.

So….. long story short. I was able to re-add the drives to their respective arrays after researching the best way to grow a mirrored array onto larger devices. Why? When the arrays degraded they did so for pending sector reads. I’ve tested the drives via smart and they seem okay. So, I tried re-adding and so far all seems good.

Anyway, here’s the link dump:

Good reminder – http://www.freddenny.com/UNIX/linuxRAID.html – making raid volumes bootable when moved.

Resizing a raid 1 system partition – http://mkfblog.blogspot.com/2007/11/resizing-raid1-system-partition.html

Looks like gparted supports software raid now – http://gparted.sourceforge.net/news.php

Just some notes for a reminder.

RAID is not a substitute for backups. If you accidentally delete a file, it’s still GONE. If you have corruption on one drive (or via the bus) your data is still GONE. Schedule backups of important data.

In sum – replacing drives in a software raid array is quite simple – especially if they’re the same size. Moving to larger drives and not wasting space is a bit more complicated and is a bridge I’ll have to cross at some point.

As a side note, I discovered that I had two more drives of equal size (2 160GB drives) in one system that I had originally intended to mirror those as well, but they needed data cleanup. I spent some time cleaning up the data on one then deleted it’s partitions – created a new partition of the linux auto raid type and create a new raid device with that drive and a missing drive. After that, I created the ext3 filesystem and moved the files over from the other drive. Then I blew away the partition on the 2nd drive and re-partitioned that to be a linux auto raid partition. Then I added it to the array and all synced well.

So, now – all of my 24/7 uptime drives are mirrored and I’ve got some research notes for how to move things around to larger drives when the time comes. I still should add a to-do item to check and see how many more sata drives I could squeeze into my desktop or server – it might be nice to have a hot spare.

Once upon a time I wrote of manually extracting a file from a mondorescue backup. Sometimes it’s just easier to do that, than have mondorescue go through 30 some cds or dvds just to find one file. (Not to mention the fact that the iso’s aren’t burned to disc but are just stored on a usb HD.) So, I’ve just had the opportunity to try to restore one of the “biggiefiles”. Mondoarchive attempts to split huge files up into smaller slices for archiving. The size is something that I think is configurable, but I haven’t made any changes from the default.

The first trick is figuring out which set of slice-0000105-0000.dat files to hunt for. Find the list of biggiefiles (usually in the archives folder called biggiefiles.txt) grep the file looking for your filename, but we will need the line number of the file. In my case I did `grep -n myfilename biggiefiles.txt` and found that the file I was looking for was at line 106.

So, I started looking and found that the slice-0000106-0000.dat file reported a different filename when I used head to read the beginning of it. So, I looked at slice-0000105-0000.dat…. aha! Our line 1 in biggiefile.txt get’s to be slice-0000000-0000.dat So we’ll always be one number off from the line number.

Okay – so in my case there were about 17 slices… ranging from slice-0000106-0000.dat to slice-0000106-0017.dat I should mention that many of these slices are bz2 compressed – so the first step is to uncompress them all bunzip2 *bz2 in the folder I was using as a temp folder for this worked.

Next, I just tried cat’ting them all together `cat slice*.dat >all_files.dat` But the resulting file was read as corrupt (I was restoring an access database. So, after a bit of looking at the mondo-archive code…. I tried something a bit different. I renamed the first slice (ending in 0000.dat) to something outside of our slice- count sequence -so I just called it outside.dat and then cat’ted the rest of them together using the same command as above. Why ? The 0000.dat file apparently is written as a header with just the filename and perhaps other information that mondoarchive uses (size?) I really don’t know what else, but I had noticed that the header on the 0001.dat was similar to the header on my real .mdb files that were floating around.

Anyway – I’m making a note of it here in case I need it again. Probably the best thing I can suggest to start with is to create a folder to work on these files outside of your backup structure. And by all means COPY instead of move the files over into it.

Sorry for the vague title, but there are various thoughts floating around at the moment. The first is a big thanks to the support at Westhost for recovering my VPS over the last weekend. I had started an automatic upgrade of wordpress in one of the sites hosted in this vps, it hung… so I ssh’ed in and found the vps was essentially ruined – it managed to wipe out quite a bit. I’m not sure if it was a coincidence or if the wordpress upgrade was really the culprit, but it managed to destroy quite a bit. The support at westhost though had things back up within 24 hours. It wasn’t just a matter of the site and database getting deleted, but files in /etc were gone (mail configuration hostname resolution wasn’t working, scp wasn’t working…. it was trashed in a bad way.)

Anyway, I’ve still been tinkering with the free morse code mp3 ringtones. In particular I’ve been tinkering with the Text to morse code mp3 generator. I’ve been trying to add utf support. The backend generator does interpret morse for utf characters, I’ve managed to get a way to decode the url encoding if they’re placed in the text box, my big challange at the moment is the file name. I have to truncate the filename so that it’s not too long, but most everything I’ve tried to truncate counts bits instead of characters. Some, even if I tell it to count characters will assume that 1 character=1bit and I wind up with some of the utf characters cut short which gives unusual filename results. In the testing version of the script I’ve just decided to sanitize things by replacing an extended character with an x….

I may spend some time in the coming weeks here giving a peek behind the curtain at some of the scripts I’ve written for that site, or for some of those that I use myself to generate a morse code mp3 from an rss feed for instance. I’m a bit reluctant to get the code of the generator script out there though for concern that I haven’t dotted all the i’s so to speak.

I have noticed in the last few weeks that java updates on windows now clean up old versions of the java runtime. (At LAST.) Java traditionally left it’s old counterparts installed making you go through and manually remove the older version. Why remove the older version you ask? Well, the java runtime engine version can be specified by java code. The default is the most recent, but if someone wanted to do something mischievous, and take advantage of a vulnerability in an older release they could specify that release and if it was installed, then it doesn’t really matter that you’ve updated.

I did notice that it failed to remove something as old as, I think, update 5 of the latest version of the java runtime engine. (Current AFAIK is update 13.) So, it would be worth a check in your control panel/add remove programs to clean out the oldest ones and then in the future it should tidy up on it’s own. (THANK YOU SUN!)

I’ve got ideas for several fairly large pages here – collections of ideas in one place. For quite some time I was pretty fond of the blog post style of entries and for many things that makes sense, but…. there is also a place for collecting lot’s of information on a single topic on a single page so that you’re less likely to “lose anything…” Sometimes I feel like the blog posts are somewhat like a large and jumbled junk drawer and the pages are neat organized boxes with labels telling exactly what is inside. And to be able to sort things up with an online
journal or blog, you need to be well equipped with the right tools and the best blog host for your site. Because sooner or later, without you noticing it, you will have a bunch of files that even you can’t imagine how you were able to have had it.

I should also mention that this site will likely be on the move soon. My webhost provider is in the midst of transitioning away from their current vps model and it sounds as though things will be heading towards more of a shared hosting style setup. This (and many of my other sites) have become complex enough in their demands that I’m thinking I’ll be moving many to a Xen vps hosted environment. I’ve used the morse code ring tones site as a first experiment of my own hosting in that xen environment and among other things I’m blown away by the responsiveness. It will (hopefully) mean a performance boost if things work out right. I haven’t finalized plans yet, but I think that’s the direction I’ll be heading. It looks as though it’s going to be a cost increase for me though as much as 50% more for my overall hosting, so…. I’ll be looking for an overall improvement in speed/responsiveness if that’s the case to at least make it worth spending more.

It seems as though there are a few other notes I was going to write here, but at the moment I can’t recall what they would be….

I’ve spent a bit of time over the last week putting some scripting together to build a new website. The site is morseringtones.com and as you might imagine there are morse code mp3 ringtones available there for download. I’ve put together a list of over 9100 names and there are 4 ringtones available for each name. These are played at 4 different speeds in morse code. For the quicker speeds the names are repeated multiple times to make sure you have a better chance to hear them if you’re using them on a cell phone. I’ve put things up to where you can submit ideas for further names if you like. I may be adding prosigns at some point in the near future and at some point may make it possible for people to go ahead and generate custom morse ringtones from a web interface. So if you want a morse code ringtone for your phone step over and take a look.

I’ve also bundled up the whole collection of them for download via lulu.com, more details can be found on the Purchase morse mp3 ringtones colleciton page.

In the future here I may detail some of the scripts that I’ve used to build this site. It’s all built on a linux server ( the generation of 9100 ringtones at 4 different speeds took about 6 hours.) I’m sure the scripts could have been designed differently but was quite proud of putting together the process for making the whole site so that future maintenance is minimal.

Anyway …-.-

Recently I had the opportunity to work on an Acer Aspire One netbook. It’s rare that I get the chance to work on a linux desktop system for a client. Most of my desktop users that I support run Windows (although I primarily run linux on my machines.) I get to do a fair amount of server level work with linux variants, but this one, being a desktop install, intrigued me.

The issue was that the buttons on the trackpad didn’t work. She also indicated that she had tried a usb mouse and those buttons didn’t work either. As I looked at it, I had already spent some time browsing through bug reports getting ideas on what might be the issue. It behaved exactly as she described and so then the real fun began.

I looked at the xorg.conf file from /etc/X11/ – it looked normal (it was the default config file) I first tried doing some edits to clearly specify the trackpad device (cursor movement with the trackpad had been working though, it was just the clicks that weren’t registering.

So, after making changes I restarted gdm (the greeter) ( /etc/init.d/gdm restart ) and gave a try. Some things started working, tapping the trackpad substituted for the left click and the right click work. The external mouse worked with both buttons.

So, I chipped away a bit more. I verified that the files existed for some of the appropriate packages, some I reinstalled. I uninstalled compiz (having read a bug report related to that) I reverted to an older version of xinput-synaptic and came back to current. I tried mapping the mouse buttons with an .xmodmap file.

Nothing productive.

One thing that I did notice was that the right trackpad or both mouse buttons would work only when I had restarted gdm, not on a freshboot. So, I started looking at the boot process and the output in /var/log/Xorg.0.log to see if there was anything significantly different being reported there. Nothing useful.

I disabled the “dontzap” feature that’s new in jaunty. If you read the release notes they have disabled the ability to “kill off” the graphical interface with ctl-alt-backspace… I essentially re-enabled that option so it’s quickly and easily possible to restart the gui.

The bottom line of what I found was this. If you’re moving the mouse (trackpad) when gdm first starts, the mouse buttons will be usable (with the exception of the left button on the trackpad (!)) If you don’t move the mouse at the startup of gdm it get’s “stuck” in pointer mode… I can bring the cursor down over the “options” menu in the lower left corner of the greeter screen and it still holds it’s i-beam icon as though you are editing a text window, the options does not highlight on the mouseover and no right/left click/tapping will work until gdm is blown away and restarts with a ctl-alt-backspace.

After that everything works with the exception of the left trackpad button (although you can tap.) I tried reversing the buttons and it’s still the one on the left that doesn’t function.

Wondering if I was missing something I tried an experiment. I (using another computer) installed Jaunty 9.04 to a USB hard drive and brought it up to date with all the current updates. I booted this usb hard drive on three or four systems and it worked marvelously (hardware detection “just worked” on each machine.) The only one to exhibit any problems was this Acer Aspire One ZG5. Guess what the problems were? well…. if I didn’t move the mouse when the greeter screen first came up I wasn’t able to do any mouse clicking (the pointer seemed frozen in the i-beam cursor icon mode) and even if I did (or restarted gdm) the left button did not work.

I came to the conclusion that there were two different issues, one I suspected may be a software issue and an update could fix it (the initial boot problem with useless buttons.) The second problem (left click not working) could just be a hardware issue – since it only affects the trackpad and not an external mouse. As I think about it, I do wonder if the trackpad were faulty (left button) could it cause the other problem when it’s first initialized (?)

If I knew I would have access to the machine again I might have filed a bug report with Ubuntu to see if they could get someone to verify this as software issue (or not). SInce I won’t have access to it, I wouldn’t be able to follow up with more information if they needed it, so instead of a bug report I’m posting here.

The only way to know for sure if it’s Ubuntu 9.04 or the netbook is to have an identical model to test, which I don’t.

So, does anyone out there have the same IDENTICAL model netbook – with jaunty 9.04 – are you seeing a similar problem or does it work just fine? The updates on this machine were current as of June 1, 2009.

Comments are open in case anyone has some input on the issue.

So, I’ve spent some time recently working with Zarafa. It’s an exchange replacement that has a completely open source version as well as a version with licenses for outlook if you want that level of integration. It’s web interface looks good and is an improvement on Exchange server’s capabilities with a non-IE browser. Anyway it can install on a linux server and integrate with several different MTA’s. Our choice was postfix since that was already installed for the purposes of processing forms on a webserver on the same host. Outgoing mail “just worked”, but postfix needed a bit of configuration to make the incoming mail work.

Our system is Ubuntu 8.04 and the original postfix setup had been a system that only sends mail and doesn’t receive. SO, instead of tinkering with the configuration files I moved them out of the way (backed them up as our working config actually.) Then I ran dpkg-reconfigure postfix to change our operating profile to that of an internet station (meaning that we would be sending and receiving mail directly.) This server is a VPS server hosted at slicehost. Anyway, the trick was configuring postfix for delivery to zarafa. I found a few conflicting suggestions, but finally settled on the following line being added to main.cf

mailbox_transport = zarafa: zarafa_destination_recipient_limit = 1

Of course, you also need to specify your domain and which domains you are receiving mail for. In this case we were going to be receiving mail for multiple domains and the following suggested addition to master.cf wasn’t going to work for us.

zarafa unix – n n – 10 pipe flags= user=vmail argv=/usr/bin/zarafa-dagent ${user}

You see, our goal was two domains with the same email frontend ….. i.e. mail@domain1.com and mail@domain2.com to be seperate accounts. By default the “user” string above was going to take our address and crop the email address before the @ sign as user… so both of those accounts would look the same… they would both be the user mail.

SO, I did a bit of digging on what variables postfix takes in it’s master.cf file. I found a bit of advice to do a “man 8 pipe” since that’s the utility that makes postfix’s delivery happen. So, finally I found a good reference in the pipe manual to do what I thought would work. We could use ${recipient} the only question was if zarafa-dagent could use that…

So as root I ran zarafa-dagent without options and got some good information including….

-R Resolve the primary email address passed in , and deliver to that user. Only one of -e and -R may be specified.

Woohoo…

So, I added this line to master.cf in place of the one suggested above…

zarafa unix – n n – 10 pipe
flags= user=vmail argv=/usr/bin/zarafa-dagent -R ${recipient}

Tested and all seems to be working well.

So, now we have abstraction of usernames and email addresses. You could have a username velma and the address jinkies@mydomain.com and delivery will still work. You can also have the username scooby and address jinkies@thisotherdomain.com and they will have unique mail stores (in spite of the first part of the address being the same.)

There are certainly situations where you would want user@domain1.com and user@domain2.com to be the same, but this is a good fix for those situations where you want to manage discrete accounts for multiple domains. From what I’ve seen thus far out of zarafa I’m liking it. Our next project will likely be imap integration

On my (somewhat older) laptop, I’ve noticed that MPlayer sometimes gets the audio and video out of sync. The audio is moving faster than the video. I’ve noticed this in octoshape streaming as well as in playing standalone videos.

I discovered a quick fix to this is to use the d hotkey on the keyboard. This toggles between frame dropping modes. By default mplayer tries to draw every frame it receives, but toggling to either framedropping enabled or framedropping hard can help keep the video up with the audio track. I’ve found this has made my octoshape streams usable again! (Usually I just make use of the framedropping enabled instead of the hard setting. (I haven’t investigated the difference between the two.)

This is for Mac or Linux. Their servers are VERY busy right now as you might imagine. But, if you’ve always wondered about crossover wine, see if you can get a copy today at Codeweavers main site. I was redirected to their DOWN page which is a lean page with the download links and a place to sign up for a license key. The license key you can register by the end of the month. Right now their server is crying since they’ve been dugg and slashdot’ted at the same time, but I’m sure it will be possible to slip in sometime today for a free copy of their polished product.

On a side note, I’ve been a paying crossover license owner for some time although on my current main laptop I’ve been using stock wine with VERY good results. If you look “up the food chain” though. Many of the good results we get in the free stock wine is thanks to the work at codeweavers. So…. in spite of todays free offer, they deserve the communities support if you believe that a working Windows compatibility layer is important to the future of operating systems such as Linux.