SED can make some things SO much easier

If you have a linux machine and haven't ever made use of sed (stream editor) you're missing out on a great automation utility. I've saved myself probably 20 hours of manual editing with about an hour of work TWICE today. Here's how.... over on the North Carolina Genealogy site I was opening forums for each county in the state of North Carolina (100 counties.) Now, I could have gone through and typed out a description, slug (address) and name for each one, but that looked too tedious. So...

I copied and pasted the list of counties to a text file (there was a carriage return after each) so I had one per line. Then I made a second file which I used tr to convert from upper case to lower case (since I wanted lower case "slugs") I then went through and used sed to convert a space to an underscore. Now, I could have done this by hand because the only county with a space in the name is New Hanover, but.... it was easy as the following...

cat county_lowercase.txt | sed 's/ /_/' > county_lowercase_nospace.txt


So, the next step was crafting the slugs...

cat county_lowercase_nospace.txt | sed 's/$/-county-nc-genealogy-queries/' > slugs.txt


I checked the forum list table for my forum software through phpmyadmin and it was a pretty simple structure, about 8 fields.... a unique id which I set to NULL in my database, forum name, description, slug, forum order, number of posts, number of topics.... I set the last two to 0 throughout (100 lines), the forum order I started a bit past the current forum order numbers that were listed in the current database and I copied and pasted from the textfile the slugs.

So, next I crafted the forum titles....

cat county_list.txt | sed 's/$/ County, NC Genealogy Query Forum/' > forum_titles.txt


And of course, those get copied and pasted into the correct column. By the way, the $ above tells sed to put the text at the end of each line so that the county name comes first.

Then the descriptions, again I cut and paste and things look good. From the spreadsheet now I export to csv and import via phpmyadmin and check, lo and behold I now have 100 forums correctly added.

The other big thing though was a script I use to retrieve rss feeds. Each page was to have one uniquely named that would be included via a php include call in the page.

So... again sed to the rescue.

#!/bin/bash
for name in `cat county_lower_nospace.txt`;
do
cat template_gen.php | sed "s/template_search/$name+county%22+nc/" > "$name"_gen.php
done

VERY easy and it worked like a charm.... 100 php scripts created, one for each county in .... 5 seconds? Nice.

Finally to top things off, I used sed to make the links to insert in to the pages... again all 100. So, it turned into a simple copy paste for each one.

Thanks sed.. I don't know how else I could have done this that didn't involve hours of menial/tedious editing.

For more information on some of the neat textfile tricks you can do with sed... take a look at this sed guide.