Sed (Stream EDitor) is a powerful Unix and Linux utility which has the ability to parse text files and make changes. It’s practically a gold mine for scripting. Fortunately, this utility has been part of the modified Red Hat Enterprise Linux service console (COS) in VMware ESX current and previous versions. Used in my deployment scripts, it serves me well for the purposes of rapidly standing up new or rebuilding existing ESX hosts.
Admittedly, I don’t have a real strong Linux background. My foundation has been primarily Microsoft Windows, and before Windows, Microsoft DOS and Commodore 64/VIC-20 BASIC programming which actually did provide me a nice basis for command line going forward. When I began building ESX hosts, I soon discovered scripting was where it’s at for efficiency and more imporantly consistency. I learned the use of sed by borrowing snippets of example code available on the internet and VMware Community forums. By the way, the sed rabbit hole goes very deep. I’ve only learned the find/replace function of it to get me by for ESX scripting which is just the tip of the iceberg.
I learned something about sed recently that probably would have helped me during my learning process of sed a while back. The majority of sed examples found on the internet (and probably in the man pages) use a forward slash / as the delimiter. This is because the / is natively used in ed, more, and vi. What I discovered is that any character can actually be used as a delimiter. This would have been helpful for me to know earlier because some of the scripting I do involves Linux paths which of course make use of the forward slash. Adding an additional forward slash for sed scripting made for an awful lot of forward slashes in one line and at times made my eyes hurt. Similar to building a complex Microsoft Excel formula whith a lot of parentheses and trying to keep track of the number of open parens versus the number of close parens.
Let me show a few examples and maybe you can see what I’m talking about.
Here’s one line of an actual script I use to add the full path display in the COS (there are three / delimiters in this example, see if you can find them):
sed -e “s/\\h \\\W/\\h \\\w/g” /etc/bashrc.old > /etc/bashrc
Forward and backwards slashes that butt up against each other are commonly reffered to as “the picket fence effect”. In the example above, for my sanity I could have chosen a different delimiter, such as an underscore, so that the script looked like this instead:
sed -e “s_\\h \\\W_\\h \\\w_g” /etc/bashrc.old > /etc/bashrc
Here’s another example which prevents the VMFS2 module from loading at startup on ESX3 hosts:
sed -e “s/echo \”vmfs2 vmfs2\”/\#echo \”vmfs2 vmfs2\”/g” /etc/init.d/vmware.old > /etc/init.d/vmware
More picket fence, not quite as bad though. Maybe this time I use use the asterisk as a delimiter so that it looks like this instead:
sed -e “s*echo \”vmfs2 vmfs2\”*\#echo \”vmfs2 vmfs2\”*g” /etc/init.d/vmware.old > /etc/init.d/vmware
So it boils down to choosing the right sed delimiter for the line of code you’re working with. Kind of like choosing the right bottle of wine for your meal, and choosing the right cigar for afterwards.
I’ve always said that much of the fun of this career I have chosen is the opportunity to learn something new every day and put it to practical use. Today has been no exception. Some days are more fun to learn than others. It depends on the conditions and circumstances…
Hello Jason,
Great post! Just a quick comment though. The yellow text doesn’t show up so well without the black background. So, it’s almost impossible to read in RSS.
Thanks for the post.
A long time ago I used sed on some data exported from a Access database to prepare it for insertion into a MSSQL database. It saved me a lot of work.
Soon after I discovered Perl. I still use sed for some tasks, but I have since used Perl for almost everything: from data wrangling, to Windows GUI apps, to Excel spreadsheet/Word document/PDF generation.
Thanks Harley – I had forgotten yellow on white in RSS readers. I’ve updated the color to blue. hopefully it is more readable.
Jas
Thanks Jason,
Looks much better now :). Also, thanks for visiting my site. It’s good to know some other Virtualization fanatics out there 🙂 .